transkribus-mcp-server

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v3.0.0
Disambiguation2/5
With 301 tools, many have overlapping purposes despite different names (e.g., multiple list/export/upload variants). Descriptions help but the volume makes disambiguation difficult for an agent.
Naming Consistency4/5
All tools follow the transkribus_category_verb_noun snake_case pattern. Minor deviations like 'du_recognize' and 'p2pala_train' exist, but overall naming is consistent.
Tool Count1/5
301 tools is far too many for a single MCP server. This overwhelms an agent and is well beyond the typical well-scoped range of 3-15 tools.
Completeness5/5
The tool surface is extremely comprehensive, covering authentication, collections, documents, pages, models, recognition, jobs, credits, search, labels, uploads, admin, and user management with full CRUD and lifecycle operations.
Average 3.2/5 across 297 of 301 tools scored. Lowest: 1.3/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 9 commits in the last 12 weeks
- Last stable release on July 17, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under Functional Source License, Version 1.1, MIT Future License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description claims 'Create or update', which is not idempotent, but annotations set idempotentHint=true. This is a direct contradiction. The openWorldHint=true suggests additional side effects, but the description fails to disclose any behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (one sentence) but sacrifices accuracy. It front-loads incorrect information, failing to clearly convey the tool's function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 5 parameters and lack of output schema, the description is grossly incomplete. It does not explain the operation, expected inputs for create vs update, or return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although parameter descriptions are present, they describe pagination and filtering fields (index, nValues, sortColumn, sortDirection) which align with a list/query operation, not create/update. The description does not explain how these parameters relate to creating or updating credit history entries, causing misalignment.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose2/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Create or update credit history entries' but the parameters (index, nValues, sortColumn, sortDirection) are typical for listing rather than creating/updating. Sibling tools transkribus_credit_create and transkribus_credit_update exist, making the purpose vague and potentially overlapping.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like transkribus_credit_create or transkribus_credit_update. No prerequisites or context are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the tool's safety is clear. The description adds no additional behavioral context, but does not contradict annotations. A 3 is appropriate as it meets the baseline without adding value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short (two words beyond the title), but it sacrifices essential information. While concise in word count, it fails to provide sufficient detail, making it underspecified rather than efficiently concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no parameters and no output schema, the description should compensate by explaining what events are, the return format, or any filtering. It does not, leaving the agent with minimal actionable information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and schema coverage is 100%. The description adds no parameter-specific meaning, but the baseline is 3 per the rules for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states it retrieves server events and notifications, which is a clear verb and resource, but it fails to differentiate from sibling tools like transkribus_server_version or transkribus_server_available_files. It is not a tautology but lacks specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is given on when to use this tool versus alternatives. There is no mention of context, exclusions, or related tools, making it impossible for an agent to decide when to invoke this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false, but description does not disclose behavior beyond duplication. No mention of side effects, permissions needed, or whether the original document is affected. With no output schema, the return value is also opaque.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is extremely short (one sentence) but under-specifies the tool's capabilities. It is not appropriately sized; it loses points for lacking necessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters, no output schema, and sparse annotations, the description is incomplete. It does not explain the duplication destination options, the effect of optional parameters, or what the response contains (e.g., new doc ID).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description 'Duplicate a document within a collection' adds no additional meaning beyond the schema's param descriptions. It does not clarify optionality of targetCollId or name default behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states verb 'Duplicate' and resource 'document within a collection', but is ambiguous about scope: it could mean only within the same collection, yet parameters allow copying to a different collection via targetCollId. It does not differentiate from sibling transkribus_coll_duplicate which duplicates the whole collection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, prerequisites, or when not to use. For example, no mention of when to use transkribus_coll_add_doc or transkribus_coll_duplicate instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate that the tool is idempotent and non-destructive. The description adds no further behavioral context, such as whether only provided fields are updated (patch behavior) or other side effects. With rich annotations, additional transparency is expected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short (one sentence) but lacks important details for a 5-parameter tool. It is under-specified rather than efficiently concise, missing opportunities to summarize behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, no output schema, but annotations present), the description fails to explain partial update behavior, return values, or the scope of metadata changes. It is incomplete for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with individual parameter descriptions. The tool description does not add any additional meaning beyond what the schema already provides, which is adequate but not enhanced.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'Update metadata for a document,' which clearly indicates the action and resource. However, it does not distinguish from the sibling tool 'transkribus_doc_update_metadata_v2,' nor does it specify which metadata fields are affected, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'transkribus_doc_update_metadata_v2' or 'transkribus_coll_update_metadata.' There is no mention of prerequisites, context, or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds no behavioral details such as pagination behavior, response format, or how filters affect results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise and front-loaded. However, it lacks substance and could benefit from more detail to be truly useful. It is minimally adequate but not well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of 22 parameters and no output schema, the description fails to provide essential context about return values, filtering semantics, or how this tool differs from similar sibling tools like transkribus_doc_list_pages.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all 22 parameters. The tool description does not add any additional meaning beyond what the schema provides, so it meets the baseline but does not elevate understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get all pages for a document,' which is a clear verb+resource pair. However, the input schema includes many filter parameters (pages, status, etc.), implying the tool can retrieve a subset, not just all pages. This mismatch makes the purpose somewhat vague and potentially misleading.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like transkribus_doc_list_pages or transkribus_doc_get_page_ids. There is no mention of prerequisites or context for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide safety profile (idempotent, non-destructive). Description adds only the method 'posting updated model data' without clarifying permissions, side effects, or what fields are updatable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise but overly minimal, lacking key details like the input body structure or expected response.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with a nested object parameter and no output schema, the description is too sparse; it does not explain what constitutes valid model data or what the update returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with description for 'body'. The tool description does not add extra meaning beyond the schema's 'Model data to update'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the verb 'update' and resource 'model', which is clear but does not specify which model or how to identify it among siblings like transkribus_model_update_by_type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as transkribus_model_delete, transkribus_model_get, or transkribus_model_update_by_type.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, destructiveHint=false, etc. The description adds only 'Get detailed statistics' without elaborating on what statistics are included or any behavioral traits (e.g., auth needs, rate limits).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with the key action. Efficient, but could include more detail without overwhelming. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and many sibling tools, the description lacks completeness—it doesn't explain what 'detailed statistics' means or what the return value looks like. Annotations partially compensate but not enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with basic descriptions for both parameters (collId, id). The description does not add any extra meaning or context beyond the schema, meeting baseline expectations.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool gets 'detailed statistics for a document,' which is a clear verb+resource. However, it does not distinguish this from sibling tools like transkribus_doc_get_stat or transkribus_doc_get_metadata, which may serve similar purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs. alternatives (e.g., transkribus_doc_get_stat, transkribus_doc_get_fulldoc). Does not provide context for selection or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate non-read-only, non-destructive, non-idempotent, and open-world behavior, but the description adds no additional context about side effects (e.g., job creation, resource modification). The description does not disclose expected changes or permissions needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, but it lacks secondary details that would aid understanding. It is front-loaded but too brief.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With nested objects and no output schema, the description is insufficient. It does not explain the effect on the collection, how results are returned, or how to monitor progress, leaving significant gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with basic descriptions for collId and body, but the description adds no further meaning. The 'body' parameter is a complex object with undefined structure; the description does not clarify required configuration fields, leaving ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs a CITlab Text2Image alignment for a collection, using specific verb 'Run' and resource 'CITlab Text2Image alignment'. However, it does not differentiate from sibling tools like transkribus_recog_text2image, which may use a different engine.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., other text-to-image tools or OCR engines). No prerequisites or context for invocation are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description claims 'download' implies a read operation, but annotations set readOnlyHint=false, suggesting possible side effects. This is a contradiction, as download typically does not modify state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at 9 words, but it omits necessary context about the tool's behavior and usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is present, and the description does not explain what a 'client file' is, what the new endpoint entails, or whether the file is from the server. The minimal information is insufficient for a download tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds no additional information beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Download' and the resource 'client file', and distinguishes from the old version by mentioning 'new endpoint'. However, it does not clarify what makes this different from other download tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus the old version or alternatives. No when-to-use or when-not-to-use information is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate write operation (readOnlyHint=false) and non-destructive (destructiveHint=false). Description adds no behavioral context beyond 'Store', such as what happens if the option already exists, authorization needs, or side effects. With annotations present, the description could still add value but does not.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the key action and resource. It wastes no words. However, it is borderline too terse; a bit more context could improve usability without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's domain (editorial declarations) and no output schema, the description is too minimal. It does not explain the relationship between options and features, the expected return value, or how the option value should be formatted. An AI agent would lack sufficient context to use this tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear descriptions for all three parameters. The description does not add any additional meaning beyond the schema. Baseline is 3 because schema already covers parameter semantics adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The title and description clearly indicate the tool creates/stores a new option for an editorial declaration feature. The verb 'Store' and resource 'new option' are specific. However, 'editorial declaration feature' is somewhat vague, but it distinguishes from sibling tools like transkribus_editdecl_post_feature which handle features rather than options.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as transkribus_editdecl_post_feature or transkribus_editdecl_delete_option. No mention of prerequisites, context, or conditions. Without this, an AI agent may struggle to choose correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the description does not need to restate safety. However, it adds no further behavioral details (e.g., pagination, default sorting, or output format). Since annotations cover the safety profile, a score of 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (4 words), which is efficient but at the expense of clarity and completeness. It is front-loaded but lacks substance needed for effective tool selection.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 10 optional parameters, no output schema, and a need to explain pagination (index, nValues) and filtering behavior, the description is incomplete. It fails to describe what the tool returns, how to use filters effectively, or how it differs from sibling list tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 10 parameters. The main description adds no additional semantics beyond what is in the schema. Baseline 3 is justified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all labels' states a verb and resource, but it is vague about scope (global vs per collection/user) and does not distinguish from sibling tools like transkribus_coll_label_list. It lacks specificity about what 'all labels' means in the context of the Transkribus system.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like transkribus_coll_label_list or transkribus_label_get. There are no usage context hints, exclusions, or mentions of prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, non-destructive, idempotent, and open world behavior. The description adds no additional behavioral context (e.g., how costs are estimated, data freshness, or error cases).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence but lacks substantive information. It does not earn its place as it fails to add value beyond the tool name and annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should explain what the tool returns. It does not mention return format, data structure, or page-range handling. Given 7 parameters and no guidance, completeness is low.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for all 7 parameters, so baseline is 3. The description adds no parameter-specific information beyond stating 'for a document', so no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it gets estimated OCR costs for a document, with a specific verb and resource. However, it does not distinguish from similar sibling tools like transkribus_doc_get_costs or transkribus_recog_get_costs, which could lead to confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., transkribus_doc_get_costs, transkribus_credit_get_costs). The description fails to set context or exclude other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide that the tool is not read-only and not destructive, but the description adds only the verb 'authorize' without detailing behavioral traits like whether it overwrites existing permissions, requires specific roles, or has side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise, but it omits important details. It is not overly verbose, but the brevity comes at the cost of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema and the presence of related sibling tools (e.g., transkribus_admin_get_job_users), the description lacks context about prerequisites, return values, or side effects. It is insufficient for an agent to use the tool confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already defines the parameters. The description adds no additional meaning beyond what the schema provides, such as explaining what a job implementation identifier or user IDs represent in context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('authorize') and the resource ('users for a job implementation'), distinguishing it from related tools like transkribus_user_is_allowed_for_job (checking authorization) and transkribus_admin_get_job_users (listing authorized users). However, it does not explicitly differentiate from these siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. There are no prerequisites or conditions mentioned, such as requiring admin privileges or ensuring the job exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the description's burden is lower. However, it adds no behavioral context beyond the annotations, such as what 'training data' includes, return format, or any restrictions. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is concise but too minimal—it fails to provide necessary context. It is front-loaded with the action, but the lack of completeness reduces effectiveness. Could be more informative without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should clarify what the training data consists of (e.g., transcripts, page IDs, or model data) and how it relates to document tasks. It does not explain the return value or behavior, leaving ambiguity given the complexity of training data in Transkribus.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers both required parameters with descriptions (100% coverage), so the description does not need to add parameter details. The description adds no extra meaning beyond what the schema provides, which is the baseline expectation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves training data for a document, but it does not explicitly differentiate it from related siblings like transkribus_doc_get_validation_data or transkribus_doc_get_test_set. The verb 'Get' and resource 'training data' are specific, but the absence of distinction from alternatives keeps it from a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as transkribus_doc_get_validation_data or transkribus_model_get_train_data. No prerequisites, use cases, or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate it is a mutation (readOnlyHint=false), but the description does not disclose details like job creation, return values, or side effects. The description adds minimal value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise but lacks structure and is overly minimal. Every word is earned but insufficient for a complex tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (10 parameters, no output schema, job execution), the description omits critical context like return type, parameter usage, and job lifecycle.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds no additional parameter meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'run' and the resource 'layout analysis on document pages'. It is specific but does not differentiate from siblings like transkribus_la_analyze_advanced.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as the advanced version or cost-related tools. No prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description simply says 'Create a new label,' which aligns with the annotation readOnlyHint=false indicating a write operation. However, it does not add any behavioral context beyond the annotations, such as side effects, required permissions, or what happens if a label with the same name exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence). However, it could include additional useful information without being verbose, so it's efficient but not optimally informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (2 parameters, no output schema), the description is incomplete. It does not explain the purpose of labels in Transkribus or relate to sibling tools, leaving the agent without sufficient context for proper usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents the parameters. The description adds no extra meaning, so it meets the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (create) and resource (label), which is specific. However, it does not differentiate from sibling tools like transkribus_label_update or transkribus_label_assign_documents, so it's clear but lacks distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as transkribus_label_update or transkribus_label_assign_documents. No prerequisites or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds minimal behavioral context beyond what annotations already provide (destructiveHint=true). It does not disclose side effects, such as whether pages are permanently deleted or unassigned, or if the label itself is affected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence and concise, but it is too minimal. It lacks structure and additional context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive operation with no output schema, the description omits important details like permanence or reversibility. It does not explain what happens to the label or pages.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds no additional meaning beyond the schema's 'Label ID' description for the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Remove all pages from a label' clearly states the verb (remove) and resource (pages from a label). It is not a tautology and distinguishes from siblings like 'remove_pages' (which likely removes specific pages). However, it could be more explicit about the 'all' vs 'some' differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'label_remove_pages' or 'label_remove_all_documents'. No prerequisites or context provided for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already supply destructiveHint=true and idempotentHint=true. The description adds no additional behavioral context (e.g., effects of removing documents not in the label, error behavior, or permissions), so it contributes minimal value beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (5 words) but lacks structure or any supplementary sentences. While brevity is appreciated, it could benefit from a bit more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's destructive nature (annotations) and the existence of sibling tools for related operations, the description omits important context such as what happens if the label or documents do not exist, or how this differs from removing all documents. The minimal information leaves gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage, clearly describing 'labelId' and 'documentIds' with types and constraints. The description does not add any extra meaning or context to these parameters, so it meets the baseline expectation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb 'Remove' and the resource 'documents from a label', making the core purpose clear. However, it does not differentiate from similar sibling tools like transkribus_label_remove_all_documents or transkribus_label_remove_pages, missing an opportunity to disambiguate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, exclusions, or context, leaving the agent to infer usage without support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint, so the safety profile is clear. The description adds no additional behavioral context, such as pagination behavior implied by index/nValues, response format, or any potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, making it concise but at the expense of completeness. It could use a more structured format (e.g., listing key parameters or behavioral notes) without increasing length excessively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of pagination parameters (index, nValues) and many similar sibling tools, the description fails to explain what training data is returned, how pagination works, or how this tool relates to others. Output schema is absent, leaving the agent without full context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all four parameters. The description only reiterates 'by type and ID' without adding semantic value beyond the schema. Baseline is 3 due to full schema coverage, and no extra meaning is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (Get), resource (training data), and key identifiers (type and ID). However, it does not differentiate from sibling tools like transkribus_model_get_train_data_docs or transkribus_recog_get_train_data, which also retrieve training data but for different scopes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With many sibling tools retrieving training data (e.g., transkribus_doc_get_train_data, transkribus_recog_get_train_data), the absence of context or exclusions makes selection ambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive behavior. The description adds no behavioral context beyond 'Get statistics' — it does not specify what statistics are returned, format, or pagination. For a read-only tool with no output schema, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no waste, but it is too sparse — every word is generic. It does not earn its place beyond stating the obvious. Could be restructured to include more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required params, no output schema, safe read), the description is barely sufficient. However, the large number of similar sibling tools demands more detail to help the agent differentiate. Missing return value info and usage examples. The description is incomplete for effective tool selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (both parameters have descriptions in the schema). The description adds no additional meaning about parameters — for example, it does not explain what 'type' values are valid or how 'id' is used. Baseline 3 is appropriate since schema does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb ('Get statistics') and resource ('training data for a model'). However, the tool name already implies this, and there is no differentiation from siblings like `transkribus_doc_get_train_data` or `transkribus_model_get_train_data`. The purpose is clear but generic.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., `transkribus_model_get_train_data` or `transkribus_model_get_train_data_docs`). No prerequisites or context on required model type or ID. The description is insufficient for an agent to decide when to invoke this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate idempotentHint=true and readOnlyHint=false, which already convey basic behavior. The description adds no details about side effects, permissions, or constraints beyond what annotations provide. No value added.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but lacks structure. It front-loads the action but misses opportunity to organize information like usage context. Could be improved without increasing length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and the description does not explain what happens upon successful update, what errors may occur, or how the alternative endpoint differs. Given the tool updates state, more context is needed for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 8 parameters. The description does not explain any parameter semantics or add meaning beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies 'Update page status' which clearly identifies the action and resource. However, it adds 'using transcript ID (alternative endpoint)' without explaining what the main endpoint is or how this differs from siblings like 'transkribus_page_update_transcript_status'. This vagueness prevents full clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this vs. alternatives. The phrase 'alternative endpoint' hints at an alternative but provides no criteria for choosing this tool over the presumed main endpoint. No when-to-use or when-not-to-use information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false and destructiveHint=false. The description adds no behavioral context beyond the fact that OCR is run. It does not disclose if the operation creates a job, is asynchronous, or has any side effects. The description does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but lacks structure. It is not front-loaded with critical information and is overly brief for a tool with 8 parameters and many siblings.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (8 parameters, no output schema, many sibling tools), the description is insufficient. It does not explain return values, job handling, or how to interpret results. The description is too sparse for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all parameters described in the input schema. The description does not add any additional meaning or context to parameters beyond what is already in the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs OCR on a document or pages. It uses a specific verb 'Run' and resource 'OCR'. However, it does not differentiate from sibling tools like transkribus_recog_run_htr_citlab or transkribus_recog_run_atr, which are also recognition tools. The name itself provides clarity but lacks sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other recognition tools (HTR, ATR, etc.). No prerequisites, context, or exclusions are provided. The description is too brief to guide an agent on appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds little beyond the annotations. 'Start training' implies a non-read-only, non-destructive action, aligning with annotations, but fails to disclose that training is an asynchronous job that may consume credits, return a job ID, or have constraints. The openWorldHint annotation suggests broad applicability, but the description provides no additional behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words, but it is overly brief. It could include additional context (e.g., 'Returns a job ID') without sacrificing conciseness. The front-loading is adequate, but depth is lacking.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is provided, and the description does not specify return values (e.g., job ID or status). Given that the tool starts a training process (likely asynchronous), the agent needs to know what to expect back. The two required parameters (collId and config) are mentioned, but the config object's complexity (nested object) is unaddressed. The description is incomplete for an AI agent to use confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%—both parameters (collId and config) have descriptions in the schema. The description adds no extra meaning; it repeats 'collection' but does not explain the config object's structure or expected keys. Baseline score of 3 is appropriate as schema already documents parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('start') and resource ('table recognition model training') with scope ('for a collection'), effectively distinguishing it from inference tools like transkribus_la_table_inference and other training tasks. However, it does not clarify what a 'collection' is in the Transkribus context, slightly limiting precision.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives such as transkribus_recog_train_htr_citlab or transkribus_recog_train_la2. No prerequisites, preconditions, or exclusions are mentioned, leaving the user to infer the appropriate use case from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotentHint=true and destructiveHint=false, so the description adds little behavioral insight. It does not explain potential side effects (e.g., index downtime, impact on searches) or what happens when different parameter combinations are used.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise but lacking important details like parameter interaction, return behavior, and usage context. It sacrifices completeness for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, so description should clarify return values or confirmation. Missing explanation of what resetting entails for different parameter combinations (all index vs. specific scope). Overall insufficient for an AI agent to confidently invoke.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter. The description only references collection, not clarifying how collId, id, and pageId interact (e.g., independent filters or hierarchical). Baseline is adequate but no added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (reset) and resource (search index), and mentions optional collection filtering. However, it does not specify that parameters for document ID and page ID are also available, leaving the scope partially ambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other search-related tools (e.g., transkribus_search_fulltext, transkribus_search_keyword). The description lacks context for when a reset is appropriate and what distinguishes it from sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotent and non-destructive behavior. The description adds only 'bulk update' but fails to explain how bulk operation works, error handling, or whether metadata is replaced or merged. With openWorldHint true, more behavioral detail is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise but lacks important details. It is front-loaded with the core action but omits context that would justify its brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given many sibling tools for metadata updates and no output schema, the description is insufficient. It does not explain the relationship to uploads, required identifiers, or expected results, making it hard for an agent to use correctly without additional assumptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add meaning beyond the schema's 'Array of document metadata objects to update'. No information about structure or required fields is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Bulk update document metadata for uploads' clearly identifies the action (bulk update), resource (document metadata), and scope (uploads). It distinguishes from sibling tools like 'transkribus_upload_bulk_update_isad_metadata' by specifying 'document metadata'. However, 'for uploads' is slightly ambiguous and could mean updates during upload process or for uploaded documents.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'transkribus_doc_update_metadata' or 'transkribus_upload_update_metadata'. No prerequisites, constraints, or context about upload state are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false, destructiveHint=false, idempotentHint=true. The description merely says 'Update metadata,' adding no behavioral details such as whether metadata is merged or replaced, or what response to expect. With annotations present, the description contributes little extra transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but lacks necessary details. It is not overly verbose, but it could be more informative without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has three parameters, no output schema, and minimal annotations, the description is too sparse. It does not explain the effect of the update (e.g., overwrite vs. merge), the expected metadata format, or any side effects, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all three parameters ('Resource ID', 'Collection ID', 'Metadata key-value pairs to update'). The description does not add any meaning beyond the schema, so it meets the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'metadata for an existing upload,' making the purpose straightforward. However, it does not differentiate from sibling tools like transkribus_doc_update_metadata or transkribus_coll_update_metadata, which also update metadata on different entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, context, or exclusions, leaving the agent without decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool as read-only (readOnlyHint=true), non-destructive, and idempotent. The description adds no additional behavioral context, such as pagination behavior or response structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently conveys the tool's purpose with no wasted words. It is front-loaded and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 5 parameters and no output schema. The description does not explain pagination (index, nValues) or sorting (sortColumn, sortDirection), and does not describe the return structure, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds no additional meaning beyond what the schema provides, resulting in a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get the credit usage history for a collection' clearly states the verb and resource, and the collId parameter implies collection-specificity. However, it does not explicitly differentiate from siblings like transkribus_credit_get_history, which may also retrieve credit history but for the user overall.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as transkribus_credit_get_history or transkribus_credit_get_transactions. The description lacks context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds no behavioral details beyond what annotations already provide (readOnlyHint, idempotentHint). It does not explain what happens with invalid collection IDs, the output format, or any safety cues. Minimal value added beyond structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently states the tool's purpose. No unnecessary words or repetition. Perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and only 2 parameters, the description should provide more context about the result format or common usage scenarios. It fails to compensate for the missing output schema, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with clear definitions for collId and legacy. The description adds no additional parameter meaning, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'tag definitions for a collection.' It is specific, but does not differentiate from sibling tools like transkribus_coll_get_metadata or transkribus_coll_label_list. A score of 4 reflects clarity without sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description lacks any context about prerequisites, conditions, or comparison to other tools for listing similar data. This is a significant gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and non-destructive, so the description's minimal addition of 'with pagination' does not significantly enhance transparency. No disclosure of performance, rate limits, or return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence with no redundancy. It could be slightly expanded to include more context without losing conciseness, but it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 24 parameters and no output schema, the description is too minimal. It lacks information about return values, default pagination behavior, and how this tool differs from other list/search tools. Annotations partially compensate, but the description remains incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. The description's mention of pagination aligns with index/nValues parameters but adds no new meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists documents in a collection and mentions pagination. However, it doesn't explicitly differentiate from sibling tools like transkribus_coll_find_documents or transkribus_coll_list_docs_xml, which have similar purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., filtered search or document counting). No contextual clues such as 'use when you need all documents without filters'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description does not disclose any behavioral traits beyond 'update'. Annotations indicate idempotentHint true, but description doesn't explain. No mention of permissions, side effects, or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff. However, it could be slightly expanded for clarity without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, nested body object lacks explanation. Annotations provide some context, but overall the description is insufficient for an agent to use the tool correctly without additional inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters (collId, body). Description adds no additional meaning beyond schema. Body is described as 'Tag definitions data' but no format or structure guidance provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates tag definitions for a specific collection. It distinguishes from sibling tools like transkribus_coll_get_tag_defs and transkribus_coll_update_default_tag_defs, though could be more explicit about the distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, prerequisites, or when not to use. Sibling tools exist for getting or updating default tag definitions, but no hints are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and openWorldHint=true, but the description adds no behavioral context beyond 'create'. It does not mention authentication needs, side effects (e.g., does it debit from a source?), or whether creation is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no unnecessary words. It is appropriately front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the four optional parameters and no output schema, the description should explain the purpose of each parameter, return value, and side effects. It is incomplete for a non-idempotent write operation with open world hint.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover all 4 parameters (100% coverage), but the tool description does not add meaning beyond what the schema already provides. The schema describes each parameter minimally; the description could clarify usage (e.g., relationship between type and amount).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb-resource combination ('Create a new credit package'), but it does not distinguish this tool from sibling credit tools like transkribus_credit_manage or transkribus_credit_handle_order, which may have overlapping functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives or what prerequisites are required. The description lacks context such as required roles, relation to other credit operations, or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate idempotentHint=true and non-destructive, but the description does not explain what processing a credit order involves (e.g., status changes, payment handling). The description adds minimal behavioral context beyond what annotations already provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise. However, it is overly brief and could benefit from additional context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's three parameters and no output schema, the description fails to explain what the result of processing is, what side effects occur, or how the parameters interact. It is insufficient for a complex operation like processing a credit order.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptive parameter names and descriptions. However, the tool description adds no additional meaning or clarifications about parameter usage or allowed values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it processes a credit order by ID. However, it does not differentiate from other credit-related tools like transkribus_credit_create or transkribus_credit_update, leaving ambiguity about what 'process' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor are there any prerequisites or conditions for use mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds minimal behavioral context beyond repeating the purpose. It does not mention rate limits, authentication requirements, or the effect of parameters like 'onlyActive' or 'userid' on the returned data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded with the key action. It is not verbose, but could be slightly more informative without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 10 parameters and no output schema, the description does not explain pagination (index, nValues), sorting, filtering, or the structure of the response. This leaves significant gaps for the agent to understand the tool's full behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameters are already documented. The tool description does not add any additional meaning or context for the parameters beyond what the schema provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and resource 'credit packages', and specifies scope 'for the current user'. However, the presence of the 'userid' parameter creates ambiguity about whether credits for other users can be listed, which is not clarified.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus sibling tools like transkribus_credit_get, transkribus_coll_credit_list, or others. The agent is left to infer usage context from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate non-readOnly, non-destructive, non-idempotent, and open-world. The description adds no behavioral details beyond 'store', such as whether duplicates are allowed or if updates occur. It does not leverage annotations to enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence of 12 words. It is front-loaded and concise, though it could include more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple creation tool with no output schema, the description is minimally adequate. However, it lacks mention of return values, side effects, or expected date formats, which would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all parameters (collId, title, date). The description adds no extra meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool stores a new milestone in the crowd project of a collection, distinguishing it from sibling tools like transkribus_crowd_post_message. However, it lacks specificity about what a milestone represents (e.g., a date with a title).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as prerequisites (e.g., existence of a crowd project) or constraints. The description does not mention conditions for use or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate idempotency and non-destructive behavior, but the description adds no behavioral context. It does not explain whether the configuration is merged or replaced, or what happens to existing settings.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with purpose. No redundant information. Efficient for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and description does not mention what the tool returns (e.g., success status, project ID). The 'body' parameter's expected structure is not elaborated. For a write operation with a complex input, more completeness is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers both parameters with descriptions (collId as 'Collection ID', body as 'Crowd project data'). However, the 'body' is an open object with no further structure hints, which the description could have addressed. Baseline 3 due to full schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (create or update) and the resource (crowd project configuration for a collection). It distinguishes from sibling read tools like transkribus_crowd_get_project, but does not differentiate between create and update scenarios.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., when to create vs update, or when to use other crowd tools). No mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false (write operation), destructiveHint=false (non-destructive), and idempotentHint=false (not idempotent), but the description adds no additional behavioral context such as side effects, permissions, or what happens if already subscribed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no wasted words. However, it could be more informative while still being concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema), the description is adequate but lacks important context about the subscription action's effects and prerequisites. The openWorldHint suggests external effects, but these are not explained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema describes 'collId' as 'Collection ID', giving basic meaning. The description does not add any further parameter semantics, but with 100% schema coverage, the baseline is 3. No extra value is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'subscribe' and the resource 'crowdsourcing project for a collection', which differentiates it from sibling tools like transkribus_crowdsource_unsubscribe and transkribus_crowdsource_list. However, it lacks specificity about what subscribing entails (e.g., notifications, participation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor any prerequisites or context (e.g., required authentication or collection ownership). The agent has no information to decide if this is the right tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds no behavioral context beyond these annotations, such as whether the data is paginated, how it is formatted, or any rate limits. The description fails to add value beyond the structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short sentence (8 words) that efficiently communicates the basic action. It is concise and front-loaded, but the extreme brevity may sacrifice necessary detail. However, within the conciseness dimension, it is appropriately sized and not verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema and the presence of many sibling tools, the description is too minimal. It does not explain what 'validation data' specifically refers to in this context, nor does it hint at the return format or structure. For a tool with moderate complexity and no output schema, the description should provide more context to aid the agent in understanding the tool's output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (both collId and id are described in the schema). The description does not add any semantic information about the parameters beyond what the schema provides. According to the rubric, when schema coverage is high, the baseline is 3 even without parameter info in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'the validation data for a document'. However, it does not distinguish this from similar sibling tools like transkribus_doc_get_train_data or transkribus_model_get_validation_data. The purpose is clear but lacks specificity to differentiate among closely related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. It does not specify prerequisites, such as requiring the document to belong to a collection or that validation data is typically associated with model training. No when-to-use or when-not-to-use information is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, implying a non-destructive write operation. The description merely says 'Run recognition', adding no detail about side effects, job creation, or output behavior. It does not exceed what annotations already convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no verbosity. However, it is overly terse and could include more detail without becoming wasteful. Still, it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the tool (recognition, many sibling tools, no output schema), the description is insufficient. It does not explain what DU recognition entails, what happens after running, or how the result is accessed. This gap makes it hard for an AI to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all four parameters. The description adds no additional parameter context beyond the schema, so it meets the baseline but does not enrich understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Run Document Understanding recognition') and objects (document, model). It identifies the specific type of recognition (DU), but does not distinguish it from sibling recognition tools like transkribus_pylaia_recognize or transkribus_recog_run_htr_citlab, which could confuse an AI agent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Given many sibling recognition tools (e.g., pylaia_recognize, recog_run_htr_citlab), the description lacks context to help an agent choose correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate non-read-only behavior. The description confirms mutation but adds no new behavioral context (e.g., side effects, required permissions, or response format). With annotations present, the description's contribution is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short (5 words) but overly vague. While front-loaded, it sacrifices necessary detail for brevity, making it less useful than a slightly longer but more informative description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has required parameters and no output schema, the description should at least hint at required inputs, return behavior, or linkage to other job tools. It lacks this context, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description does not add any parameter-specific meaning beyond the schema, such as explaining what 'type' values are expected or how docId/collId relate to job creation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb and resource ('Create a new processing job'), but does not distinguish from sibling tools like transkribus_job_update or transkribus_job_kill, missing an opportunity for differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., transkribus_job_update for modifying jobs, transkribus_job_list for listing). No context on prerequisites or appropriate scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds no behavioral context beyond 'get details', such as what fields are returned, error conditions, or rate limits. For a tool with rich annotations, more context would be beneficial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that communicates the essential purpose without extraneous words. No wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema, the description should elaborate on what 'details' includes (e.g., status, progress, creation time). The tool is relatively simple, but the description leaves ambiguity about the return structure, which is important for an agent to decide if this tool meets its needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage for parameters is 100%, and the description does not add additional meaning to the 'id' parameter beyond what the schema already provides. Baseline of 3 is appropriate since the schema handles the semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool retrieves details for a specific job using its ID. It differentiates from sibling tools like 'transkribus_job_list' and 'transkribus_job_count' by specifying it is for a single job, but does not explicitly contrast with other get-type tools like 'transkribus_job_get_errors'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates basic usage: get details by ID. However, it provides no guidance on when to use this tool versus alternatives (e.g., job_list for summaries, job_get_errors for error details) or any prerequisites or limitations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only says 'assign' without detailing behavioral traits beyond what annotations provide. Annotations indicate openWorldHint=true but provide no specifics on side effects, permissions, or whether assignments are cumulative. The description adds no meaningful behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is as concise as possible, though it sacrifices completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool modifies page-label assignments, the description lacks important context such as whether assignments are added or replaced, what happens if the label or pages are invalid, and any prerequisites. The annotations and schema do not compensate for this missing information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the input schema already describes both parameters (labelId and pageIds). The description adds no additional semantic value beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (assign) and resources (pages, label). It distinguishes from sibling tools like transkribus_label_assign_documents by focusing on pages rather than documents, but does not differentiate from transkribus_label_replace_page_assignments.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as transkribus_label_replace_page_assignments or transkribus_label_remove_pages. The context of usage (e.g., additive vs. replacement) is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds minimal behavioral context; 'replace' implies overwriting existing assignments, but it does not clarify whether all prior assignments are removed or how the combination of labelId and labelIds works. No details on side effects or authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no filler. However, it is perhaps too brief to convey necessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists. The description does not explain the behavior of replacement (e.g., whether it removes existing labels before assigning new ones), the effect on the document, or error conditions. Essential context is missing for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but schema descriptions are minimal ('Label ID', 'Resource ID', 'Array of label IDs to assign'). The description adds no extra meaning and fails to explain the ambiguous relationship between labelId (required) and labelIds. This does not compensate for the schema's lack of clarity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Replace label assignments for a specific document' uses a specific verb ('Replace') and identifies the resource ('label assignments' for a document). It clearly distinguishes from sibling tools like transkribus_label_assign_documents (assign) and transkribus_label_remove_documents (remove).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., assign or remove labels). No prerequisites or context are provided, leaving the agent to infer usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a mutation (readOnlyHint=false) and non-destructive (destructiveHint=false), but the description adds no additional behavioral context. It does not mention side effects, required permissions, or what happens if the collection is already associated with the model.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. However, it is somewhat sparse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description omits success indicators, error conditions, and prerequisites (e.g., model must exist). Given the context signals (3 required params, no output schema), the description is insufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions. The description merely restates the purpose of type and modelId without adding new meaning. Baseline score 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add a collection to a model') and identifies key parameters ('by type and model ID'). However, it does not differentiate from siblings like transkribus_model_add_collection_by_id, which likely serves a similar purpose with a different identifier.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus related siblings (e.g., transkribus_model_add_collection_by_id, transkribus_model_remove_collection). The agent receives no contextual advice on prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. The description adds minimal context beyond stating it gets training data documents, failing to elaborate on behavior like pagination or output format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters and no output schema, the description should explain pagination (index, nValues) and the nature of returned documents. It is too brief to be fully informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover 100% of parameters, so the description does not need to add much. It provides no additional semantics beyond what the schema already offers, resulting in a baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves documents used as training data for a model, using a specific verb and resource. However, it does not distinguish from sibling tools like transkribus_model_get_train_data, which may also return documents.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool over alternatives such as transkribus_model_get_train_data or transkribus_model_get_train_data_stats. The description lacks context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, and non-destructive behavior. The description adds no additional behavioral context, such as that the tool returns aggregated statistics without side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single concise sentence with no wasted words. Appropriate for a simple tool, though lacks structured sections.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Description adequately conveys the tool's purpose but does not mention return format or any conditions (e.g., what happens if validation data is missing). Absence of output schema increases need for more detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both parameters (id and type) fully described. The description adds no extra meaning beyond the schema for these parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool retrieves statistics about validation data for a model, distinguishing it from related tools like get_validation_data or get_validation_data_docs. However, it does not specify what types of statistics are returned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as transkribus_model_get_validation_data or transkribus_model_get_train_data_stats. Missing prerequisites or context (e.g., model must exist, validation set must be present).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, but description adds no additional behavioral traits such as pagination, sorting, error handling, or what 'collections' includes. Minimal added value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no unnecessary words. Could be slightly more informative but is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with two parameters and annotations covering safety, it is adequate. However, lacking output schema, it would benefit from saying what the response contains (e.g., list of collection IDs/names).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description does not add meaning beyond the schema, e.g., explaining how 'transcriptIds' affects the output. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'List' and resource 'collections associated with a model by its ID'. The name itself is descriptive, but does not explicitly differentiate from siblings like 'transkribus_model_list_collections' which may have different parameters.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives such as 'transkribus_model_list_collections' or 'transkribus_model_get'. No mention of prerequisites or when 'transcriptIds' should be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds no behavioral details beyond stating 'Get details', so it does not enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 8 words, which is concise and front-loaded. However, the brevity sacrifices specificity; a bit more detail on return values would improve clarity without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema and the description is vague about 'details', the description is incomplete. It does not clarify what the tool returns, which is critical for an agent to understand its capability.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions already present. The tool description does not add any additional semantic value beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly indicates the action ('Get') and resource ('page'), but does not specify what 'details' refers to, leaving ambiguity compared to sibling tools like transkribus_page_get_metadata or transkribus_page_get_plaintext which have more specific purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Sibling tools include many specialized getters, and the description gives no context for choosing this one.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnlyHint and idempotentHint annotations, the description adds no behavioral details. It doesn't explain what a 'transcript version' entails, how versions are differentiated, or the output format. With annotations covering safety, the description still lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence. While it's short, it is not overly terse for a simple read operation. Could benefit from additional structure but is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool retrieves a specific transcript version but lacks information on what the response contains (no output schema). With four required IDs and no versioning context, the description is minimally adequate for a read operation but incomplete for complex scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage, explaining each parameter (collId, id, page, transcriptId). The description does not add further meaning beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'a specific transcript version for a page'. It distinguishes from siblings like 'transkribus_page_get_curr_transcript' by implying version specificity, but does not explicitly contrast with other page transcript tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'transkribus_page_get_curr_transcript' or 'transkribus_page_get_text'. The description omits context for usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds no behavioral context beyond the annotations. With annotations providing readOnlyHint=false, destructiveHint=false, idempotentHint=false, and openWorldHint=true, the description merely restates the purpose without explaining side effects, auth needs, or what 'add' entails (copy vs. link).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded and contains no extraneous words. It is appropriately concise for a simple operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters and no output schema, the description lacks context about return values, error behavior, and how it relates to similar sibling tools. This incomplete information may hinder an agent's correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the input schema already clearly describes each parameter. The description adds no extra meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Add) and the resource (recognition model) and target (collection). It is specific enough to distinguish from generic model tools, though it does not explicitly differentiate from sibling tools like transkribus_model_add_collection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus siblings such as transkribus_model_add_collection or transkribus_recog_remove_from_collection. No prerequisites or conditions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, but the description adds no behavioral details beyond what annotations provide. It does not explain whether the computation is synchronous or asynchronous, whether it returns results directly, or any side effects. The openWorldHint=true suggests unknown behavior, but the description does not clarify.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is well-structured and concise. It conveys the essential purpose without excess words. However, it might be too brief, missing some important details. It earns points for efficiency but loses some for under-specification.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema, so the description should hint at what is returned or the nature of the output. It does not. Additionally, the compute action's behavior (synchronous/asynchronous, blocking, etc.) is not mentioned. Given the complexity and lack of output schema, the description is incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all parameters with descriptions, so schema_description_coverage is 100%. The description does not add any additional meaning beyond what the schema provides. The 'body' parameter is vague, but the schema describes it as 'Additional error rate computation parameters'. A score of 3 is appropriate as the schema does the work, but the description adds no extra insight.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Compute the error rate for a model in a collection', which is specific about the action and resources. However, it does not specify the type of error rate (e.g., CER, WER), which could be important for distinguishing from sibling tools like compute_wer. It does distinguish itself from compute_wer and compute_wer_tasas by using a more general term.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as compute_wer or compute_sample. It does not mention prerequisites, context, or conditions for usage. The absence of any usage guidelines makes it hard for an agent to decide when this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false and destructiveHint=false, so the description's 'compute' aligns with a non-read action. However, the description adds no extra behavioral details—such as what a 'recognition sample' entails, whether it creates persistent data, or any side effects beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no waste, but it is too brief to be optimally helpful. For a tool with only two parameters, it could include a bit more context about what 'compute a recognition sample' produces or requires without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal: no output schema exists, and the description fails to explain what a recognition sample is, what the tool returns, or any side effects. Given the openWorldHint annotation, the tool may have additional undocumented behavior, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with basic descriptions for collId ('Collection ID') and docId ('Document ID'). The description does not add any additional meaning or context about these parameters, so it provides no value beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'compute' and the resource 'recognition sample' for a document in a collection, which is specific and actionable. However, it does not differentiate from sibling tools like transkribus_recog_compute_error_rate or transkribus_recog_run_htr_citlab, all of which involve computing something recognition-related.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The sibling list includes many recog_* tools, but the description gives no context on scenarios, prerequisites, or why a user would choose 'compute sample' over 'compute error rate' or 'run HTR'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, indicating a safe read-only operation. The description adds no further behavioral context beyond the stated purpose, which is acceptable given annotation coverage but misses an opportunity to clarify return values or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (5 words). While not verbose, it under-specifies the tool's behavior given 6 parameters and no output schema. It could be slightly longer to improve clarity without being redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should at least mention the return type or format. It does not address what the tool returns. Additionally, the complex parameters (ref, hyp, key) are not explained in context. The rich annotations partially compensate, but overall completeness is lacking.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all 6 parameters. The description does not add any extra meaning beyond what the schema provides. Baseline score of 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Compute) and the resource (Word Error Rate for a document). However, it does not differentiate from sibling tools like transkribus_recog_compute_error_rate or transkribus_recog_compute_wer_tasas, which could cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., compute_error_rate, compute_wer_tasas). The description lacks context for usage scenarios or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true, but the description adds no behavioral context beyond stating the deletion action. It does not explain irreversibility, required permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single clear sentence with no unnecessary words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the rich annotations (destructive, idempotent) and no output schema, the description is minimal but lacks completeness. It does not address that deletion is permanent, potential dependencies, or how to recover. For a destructive tool, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions for both parameters ('Collection ID' and 'Resource ID'). The description adds no additional meaning beyond what the schema provides, meeting the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete), resource (recognition model), and scope (from a collection). It distinguishes well from similar tools like transkribus_model_delete and transkribus_recog_remove_from_collection, though it does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, prerequisites, or situations where it should not be used. This is a significant gap for a destructive operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the description's claim of 'Get information' is consistent. However, it does not elaborate on what specific information is returned, potential prerequisites, or any side effects. The description adds minimal behavioral context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with 8 words, which is efficient. However, it may be too terse and could benefit from additional structure or details. No wasted words, but under-specification slightly reduces score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description should ideally explain what information is returned when adding a recognition model to collections. The current description is minimal and does not provide enough context for an AI agent to fully understand the tool's output or capabilities.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters described ('Collection ID' and 'Resource ID'). The description does not add any additional meaning beyond the schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get information about adding a recognition model to collections.' It uses a specific verb and resource. However, it does not distinguish itself from sibling tools like transkribus_recog_get or transkribus_recog_add_to_collection, which could cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description does not provide context for when to choose this over related tools such as transkribus_recog_add_to_collection or transkribus_recog_list_collections. An explicit 'when-not' or mention of alternatives is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not add any behavioral context beyond what is already provided by annotations. Annotations indicate read-only, idempotent, and non-destructive behavior, but the description offers no additional insights such as authentication requirements or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded. It conveys the essential purpose without any unnecessary words, earning its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of many sibling tools and the absence of an output schema, the description is too sparse. It does not explain what 'cost information' entails, how it relates to the parameters, or how the response should be interpreted, leaving the agent under-informed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for all 5 parameters, so the baseline is 3. The description does not add any further explanation or meaning to the parameters beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get the cost information for a recognition model', which clearly identifies the verb and resource. However, it does not differentiate from sibling tools like transkribus_credit_get_costs or transkribus_doc_get_costs, which also get cost information for different entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of context, prerequisites, or exclusions, leaving the agent to infer from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds no behavioral context beyond stating it 'Get's data, missing details like data format, pagination, or error conditions that would aid the agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence. It is front-loaded with the key action and resource, but omits useful details that could be included without excess verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters and no output schema, the description lacks completeness: it doesn't specify what the test set data contains, how results are returned, or how it relates to recognition models. Sibling tools like transkribus_recog_get_train_set suggest there are distinct datasets, but the description does not provide this context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not elaborate on the parameters (collId, id, nrOfTranscripts) or their meaning for this specific tool, adding no value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the verb 'Get' and resource 'test set data for a recognition model', making the purpose clear. It differentiates from document-level test set tools like transkribus_doc_get_test_set, but does not explicitly contrast with sibling tools like transkribus_recog_get_train_set, so it lacks full differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no context on prerequisites or exclusions. The description is too minimal to inform usage decisions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, which cover safety and idempotency. The description adds no additional behavioral details (e.g., return format, pagination behavior, or required permissions), providing minimal extra value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It is as concise as possible while conveying the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters and no output schema, the description lacks completeness. It does not explain what 'training data' comprises, how pagination works (index/nValues), or what the output looks like. The tool's role relative to model training is also unclear, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with basic descriptions for all four parameters (collId, id, index, nValues). However, the description does not enrich parameter semantics or clarify their usage, so it does not exceed the baseline for full schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'training data' for a 'recognition model'. It is specific enough to convey the tool's purpose, but does not differentiate it from similar sibling tools like transkribus_recog_get_train_set or transkribus_model_get_train_data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. It does not mention context, prerequisites, or situations where this tool is preferred over similar tools, leaving the agent without decision-making information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already indicate readOnly, non-destructive, idempotent, and openWorld. The description adds only 'with optional filtering', which is already implicit from the schema parameters. No additional behavioral context is provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that captures the core function. However, it could be slightly more informative without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and 9 optional parameters, the description is somewhat incomplete. It does not mention pagination (index, nValues), the nature of returned data, or the significance of filtering parameters. Annotations provide safety guarantees but not operational context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all 9 parameters, so the baseline is 3. The description adds no further meaning beyond stating 'optional filtering', which does not enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'List' and the resource 'recognition models', with 'optional filtering' adding specificity. It distinguishes from siblings like transkribus_recog_list_by_collection which list models within a specific collection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_model_list or transkribus_recog_list_by_collection. The description does not specify prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate it is not read-only (readOnlyHint=false) and not destructive (destructiveHint=false). The description confirms it 'runs' an alignment, implying a job is started, but it does not disclose potential side effects, asynchronicity, or that it requires credits. The description adds minimal behavioral context beyond what annotations imply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, very concise (10 words), and front-loaded with the key action and resource. However, it could include more detail without becoming verbose. It earns a 4 for efficiency, not penalized for lack of completeness in this dimension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, no output schema, open-world hint, many siblings), the description is severely lacking. It does not explain what 'Text2Image alignment' entails, how the 'body' parameter configures it, what the return value is (likely a job ID), or how credits are consumed. The description is insufficient for an agent to use this tool correctly without additional knowledge.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the input schema already describes all parameters (e.g., collId, modelId, body, credits, doNotDeleteWorkDir). The description does not enhance understanding of what these parameters mean or how they interact. Baseline 3 is appropriate since the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Run) and resource (Text2Image alignment) and context (using a specific model in a collection). However, it does not differentiate this tool from sibling tools like transkribus_recog_text2image_citlab or transkribus_recog_text2image_matching, which have similar names and likely similar purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus its siblings. There is no mention of prerequisites, appropriate contexts, or circumstances where this tool should be avoided. The description merely states what it does without helping the agent decide when to invoke it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate the tool is not read-only, not destructive, not idempotent, and open world. The description's single phrase 'Run Text2Image matching' does not disclose any behavioral characteristics beyond what annotations already imply—such as whether it creates a job, how long it takes, or error conditions. It adds no meaningful context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is direct and concise. It wastes no words, but it is not structured with bullet points or sections. However, for a simple tool, the conciseness is appropriate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 15 parameters and no output schema, the description is too brief. It does not explain what the matching process entails, what the output looks like, or how to interpret results. The tool is complex, and the description lacks necessary details for an agent to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 15 parameters have descriptions in the input schema (100% coverage). The description does not add any extra meaning or context beyond what the schema provides. Baseline is 3 because schema coverage is high, but the description fails to elaborate on parameter usage or relationships.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Run') and the resource ('Text2Image matching') with the context of using a specific model in a collection. However, it does not distinguish this tool from similar sibling tools like 'transkribus_recog_text2image' or 'transkribus_recog_text2image_citlab', which share the same core resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There is no mention of prerequisites, when not to use it, or comparison to other tools. The agent is given no context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds no behavioral context such as pagination, result format, or rate limits. For a tool with many parameters, more transparency would be helpful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words, making it concise and front-loaded. However, it might be too terse given the tool's complexity (9 parameters, multiple siblings).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 9 parameters and no output schema, the description is incomplete. It does not explain how keyword search works, how parameters interact, or what the response looks like. The annotations cover safety but not usage details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all parameters. The description does not add any additional meaning or context for the parameters, which is acceptable given high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches for documents by keyword, with a specific verb and resource. However, it does not explicitly differentiate from sibling tools like transkribus_search_fulltext or transkribus_search_tags, which is a minor gap.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites or context for use. The single sentence provides no usage direction beyond the basic purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate mutation (readOnlyHint=false) but description adds no detail about scope, reversibility, or side effects beyond 'replace across documents'. For a mutation tool, more behavioral context is expected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence conveys core purpose efficiently, though additional details could be incorporated without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Description lacks explanation of scope (e.g., which documents are affected) and return value. With no output schema, the agent needs hints about what the tool returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter descriptions. Description adds no extra semantics beyond what is already in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states search and replace action across documents. However, it does not differentiate from sibling tool transkribus_search_replace_in_collection, which likely has similar purpose but scoped to collections.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., transkribus_search_replace_in_collection). No prerequisites or limitations mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds no behavioral context beyond the annotations, such as what happens if the collection does not exist or permission issues. With annotations covering the safety profile, a score of 2 is appropriate as the description provides no additional transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is front-loaded and efficient. However, it might be considered too brief given the complexity of the tool, but it earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the high number of sibling tools and lack of output schema, the description is insufficient. It does not explain what statistics are returned (e.g., document count, size) or the format. While minimal, it lacks completeness for an agent to use effectively without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage for the single parameter collId with description 'Collection ID'. The tool description does not add any further semantic meaning for the parameter. Baseline 3 is correct since the schema already documents the parameter adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get statistics for a collection' clearly states the verb and resource, distinguishing it from sibling tools like transkribus_coll_get_metadata or transkribus_coll_list. However, it does not specify what type of statistics (e.g., counts, sizes), which would enhance clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. For example, it does not clarify that this tool is for aggregate statistics rather than detailed metadata retrieval, or when to use transkribus_stat_get_coll_stats instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds no behavioral context beyond that, missing opportunities to describe the returned data or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is concise and front-loaded with the verb 'Get'. It effectively communicates the core action without extraneous content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters, no output schema, and a technical domain, the description is too minimal. It does not explain what 'page status overview' entails, what the return format is, or how parameters like index/nValues affect results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all 5 parameters. According to guidelines, baseline is 3. The description does not add parameter-specific info, but the schema already sufficiently documents them.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get the page status overview for a collection' clearly states the verb, resource, and context. However, it does not differentiate from sibling tools like transkribus_stat_get_coll or transkribus_stat_get_pages_filtered, which reduces clarity for selection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description lacks any context about prerequisites, typical use cases, or exclusion criteria, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds no behavioral context beyond what annotations provide, such as response format or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short sentence that is front-loaded and concise. It efficiently communicates the core purpose without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema and a vague description. It fails to explain what 'storage usage details' means or what the response contains. Given the low complexity, the description should provide more context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 1 parameter with description 'Collection ID', and schema coverage is 100%. The tool description repeats 'for a collection' but adds no additional meaning or constraints beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get'), resource ('storage usage details'), and scope ('for a collection'). It distinguishes from sibling tools like 'transkribus_stat_get_coll' and 'transkribus_stat_get_coll_stats' by specifying 'storage usage' vs general stats, though not explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as 'transkribus_stat_get_coll' or 'transkribus_stat_get_coll_stats'. The description does not mention prerequisites, exclusions, or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description merely says 'Update', which aligns with the write operation implied by annotations (readOnlyHint=false). However, it adds no behavioral details such as allowed status values, side effects on other pages, or whether the operation is scoped to a single page (pageId parameter is optional). The annotations provide idempotentHint=true and openWorldHint=true, but the description does not explain these or their implications. The lack of transparency about valid status inputs and the effect of omitting pageId is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is appropriately front-loaded and efficient. However, it could be slightly improved by integrating key limitations or clarifying scope without adding bulk.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is no output schema and the tool has 3 parameters (one optional), the description is too sparse. It does not explain what status values are acceptable, whether updating applies to all pages in the collection when pageId is omitted, or what the expected behavior is. The name suggests a statistics-related update, but the context is unclear. The presence of sibling tools like 'transkribus_stat_get_page_status' suggests a related query tool, but no linkage is provided.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters. The description does not add any additional semantic meaning or examples beyond what the schema provides. Therefore, a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the verb 'Update' and the resource 'status of pages in a collection', making the core action clear. However, it does not differentiate from sibling tools like 'transkribus_page_update_status_v2' or 'transkribus_page_update_transcript_status', which may cause confusion about which tool to use for specific page status updates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. It does not mention prerequisites, typical use cases, or situations where the tool should be avoided. The agent receives no help in decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a non-read-only, non-destructive creation action, but the description adds no behavioral details beyond 'create upload'. No mention of side effects, return value, or asynchronous behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, minimal and to the point. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, so description should hint at result (e.g., upload ID). Lacks information for an agent to understand what the tool returns or how it fits into a workflow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema already has clear descriptions for both parameters (collId, metsUrl). The tool description does not add extra meaning, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the action 'Create upload' and resource 'from METS file', which is specific. However, it does not differentiate from sibling tools like transkribus_coll_create_doc_from_mets or transkribus_upload_create_s3, leaving potential confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., transkribus_upload_create_s3, transkribus_coll_create_doc_from_mets_url). Missing context about prerequisites or workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false (implying mutation) and openWorldHint=true, but the description does not clarify what happens when inference is run (e.g., does it modify page data, create a job, or return results?). No behavioral disclosure beyond the minimal purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, concise, but lacks structure. Could benefit from additional context about usage or behavior while remaining brief.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description does not explain return values or side effects. With 5 parameters and no guidance, the description is inadequate for an agent to fully understand the tool's operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage with individual parameter descriptions, so the description does not need to add parameter details. Baseline 3 is appropriate; no extra value provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states a specific action: 'Run table structure inference on document pages.' It uses a specific verb ('run') and resource ('table structure inference' on 'document pages'), distinguishing it from sibling tools like transkribus_la_analyze which suggests general layout analysis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as transkribus_la_analyze or transkribus_la_la2_inference. Prerequisites, required model, or typical use cases are not mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate non-read-only, non-destructive, non-idempotent, and open world. The description adds no behavioral information beyond these annotations, such as duration, side effects on collection, or that training creates a new model. The lack of additional context leaves the agent uninformed about the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but lacks important details such as the return behavior or the nature of the 'config' object. It is front-loaded with purpose but insufficiently informative for a complex training tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and the complexity of starting a training job, the description should mention that it triggers an asynchronous job, what the return value indicates, or how to monitor progress. The current minimal text leaves significant gaps for an agent to correctly use the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with 'collId' described as 'Collection ID' and 'config' as 'Training configuration parameters'. The description adds no extra meaning beyond the schema, so it meets the baseline but does not enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Start CITlab HTR model training' for a collection, with a specific verb and resource. It distinguishes from siblings by naming 'CITlab', which differentiates it from other training tools like 'transkribus_pylaia_train' or 'transkribus_recog_train_la2'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when or when not to use this tool, nor does it mention prerequisites or alternatives. There is no context about required data states, collection readiness, or comparison with sibling training tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false, which aligns with the description's implication of a write operation. However, the description adds no behavioral details beyond annotations, such as side effects, required permissions, or what constitutes an 'upload'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but perhaps overly minimal. It front-loads the action but lacks structure and additional context that could be included without verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description does not explain the return value or the process after creation. The openWorldHint suggests unknown side effects, but the description fails to provide completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema documentation coverage is 100%, so the baseline is 3. The description does not add any additional meaning or examples for the parameters (collId, id, s3Bucket, s3Key, title).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create an upload') and specifies the source ('from an S3 source'). This distinguishes it from sibling upload creation tools like 'transkribus_upload_create_from_mets' and 'transkribus_upload_create_structure'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor are there prerequisites or context about the upload process. The description is purely declarative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, so the description adds minimal behavioral context beyond stating optional filtering. It does not describe the response format, pagination behavior, or any side effects, but annotations cover the safety profile.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 12 words, extremely concise. It is front-loaded with the core action and filtering options. While concise, it could be more structured by separating pagination detail, but there is no wasted content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 17 parameters, no output schema, and no explicit return value description, the description is too minimal. It does not explain the meaning of 'actions', the response structure, or pagination behavior. Sibling tools exist but no guidance is provided. The description leaves significant gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (all 17 parameters have descriptions). The description mentions filtering by collection, type, and pagination, which maps to a subset of parameters, but does not explain pagination-related parameters like index, nValues, sortColumn, etc. Baseline 3 is appropriate as the schema already provides parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists actions with optional filtering, using a specific verb 'List' and resource 'actions'. However, it does not differentiate from sibling tools like transkribus_action_count or transkribus_action_info, which perform related but different tasks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention when not to use it or any prerequisites. The agent must infer usage from the name and schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, so the agent knows it is a safe, non-destructive read. The description adds minimal extra behavioral context beyond confirming it retrieves log output. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that is front-loaded and concise. Every word earns its place, though it is very brief.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the number of sibling tools and the lack of an output schema, the description is too minimal. It does not explain the content of the log output, expected format, or any constraints (e.g., job must exist), leaving the agent with insufficient information to fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear field descriptions ('Resource ID', 'Database identifier'). The description does not add any additional meaning to the parameters, so a baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Retrieve' and the resource 'log output for a specific job', making the purpose evident. It is distinct from sibling tools like transkribus_job_get (which gets job details) and transkribus_job_get_errors (which gets errors), though it does not explicitly differentiate itself.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as transkribus_job_get or transkribus_job_list. It lacks context about prerequisites, job state, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, idempotentHint=true. The description adds no additional behavioral context such as side effects, authorization needs, or what happens if the package doesn't exist. It is consistent with annotations, so no contradiction, but does not enhance information beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is appropriately sized for a straightforward operation, though a bit more detail on the package removal process would not hurt.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists. The description does not indicate what the tool returns upon success or failure, nor does it mention any side effects or error conditions. For a removal tool, this is a significant gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions for both parameters ('Collection ID', 'Credit package ID to remove') are clear and cover 100% of parameters. The tool description adds no further meaning beyond the schema, meeting the baseline expectation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action and resource: 'Remove a credit package from a collection.' It distinguishes from sibling tools like 'add' and 'list' by using 'remove', though it could be more specific about the context (e.g., removing a previously added package).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., credit removal via other methods), no prerequisites, and no conditions for safe usage. The description is purely declarative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the safety profile is clear. The description adds no further behavioral context, but does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise and front-loaded. It is appropriately brief for a simple read tool, though it could slightly elaborate on parameters without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 5 parameters and no output schema, the description should mention the return format (e.g., list of transactions, pagination behavior). It does not, leaving the agent guessing about what the tool returns.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter has a description. The tool description provides no additional meaning beyond the schema. Baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'get' and the resource 'credit transaction records' for a collection. It distinguishes from generic credit transaction tools but does not explicitly differentiate from siblings like transkribus_coll_credit_history or transkribus_coll_credit_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus siblings such as transkribus_credit_get_transactions, transkribus_coll_credit_history, or transkribus_coll_credit_list. The agent receives no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate non-destructive, idempotent, and open-world behavior, but the description adds no additional behavioral context (e.g., whether it replaces or merges definitions, or any side effects). It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded and contains no wasted words. However, it could be slightly more informative without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool that updates default tag definitions globally, the description lacks details on the expected structure of the 'body' parameter, the impact on existing definitions, and any prerequisites or consequences. More context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters, with 'body' described as 'Tag definitions data'. The description adds no further semantic value beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and the resource ('default tag definitions across all collections'). It distinguishes from sibling tools like transkribus_coll_update_tag_defs by specifying 'across all collections', implying a global scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool vs. alternatives (e.g., transkribus_coll_update_tag_defs for per-collection updates). No prerequisites or context are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide basic info (not readOnly, not destructive, openWorld), but the description adds no additional behavioral context. Critical aspects like file size limits, overwrite behavior, or response format are absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, front-loading the key action. It could be slightly expanded with essential details without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite annotations and high schema coverage, the description omits critical information about the multipart upload (e.g., what file parameter is expected, how to provide it). The absence of a file parameter in the schema makes the description misleading.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with basic descriptions. The tool description adds no extra meaning beyond the schema, such as constraints on title/author or the multipart specifics for file upload.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('upload'), the resource ('document to a collection'), and the method ('multipart form data'). It distinguishes from sibling tools like transkribus_coll_upload_doc by specifying the upload mechanism.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no guidance on when to use this tool versus alternatives (e.g., transkribus_coll_upload_doc, transkribus_coll_create_doc_from_pdf). It does not specify prerequisites, file types, or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate mutation (readOnlyHint=false) and non-destructiveness. The description adds no additional behavioral details, such as whether adding an existing user results in an error or is idempotent, or any authorization requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence of 10 words. It is front-loaded and to the point. However, it could include a bit more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of sibling tools and no output schema, the description is incomplete. It does not explain return values, error handling, or behavior when adding a user that already exists. Annotations are present but underutilized.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all parameters. The description highlights that role is optional but adds no new information beyond what the schema provides. Minimal extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add'), the resource ('user'), and the target ('collection'), with an optional role. This distinguishes it from sibling tools like transkribus_coll_user_remove and transkribus_coll_user_add_or_modify, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like transkribus_coll_user_add_or_modify or transkribus_coll_user_update_role. There is no mention of prerequisites, when not to use, or any edge cases like adding an existing user.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only repeats the basic purpose, adding no behavioral details beyond what annotations already convey (read-only, idempotent, etc.). It does not mention pagination, sorting behavior, or what information is returned about users.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that efficiently conveys the tool's purpose with no extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity (6 parameters, no output schema), the description is minimal. It does not describe the return format or pagination defaults, which would be helpful. The openWorldHint annotation somewhat mitigates this, but more detail would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides descriptions for all 6 parameters (100% coverage), so the description does not need to add parameter semantics. The baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves a list of users with access to a collection, using a specific verb ('Get') and resource ('list of users'). However, it does not differentiate from the sibling tool 'transkribus_coll_user_list', which likely has a similar purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'transkribus_coll_user_list' or 'transkribus_coll_user_count'. The description lacks context for appropriate usage scenarios or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, non-destructive, idempotent, and open-world behavior. The description adds no additional behavioral context, such as pagination behavior or result structure, which is acceptable given the annotations but could be improved.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. However, it is slightly under-informative given the complexity of the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is too brief for a tool with 5 parameters, no output schema, and numerous sibling tools. It does not explain the purpose of pagination parameters or what statistics are returned, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 5 parameters are fully described in the input schema (100% coverage), so the description adds no additional meaning. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves user statistics for a collection, using the verb 'Get' and specifying the resource. It distinguishes from similar tools like transkribus_coll_user_list (which lists users) but could be more specific about the type of statistics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as transkribus_coll_user_count or transkribus_coll_user_get_list. The agent is left to infer from the name and context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false (write operation), destructiveHint=false (not destructive), and openWorldHint=true (side effects). The description adds little beyond stating it 'stores' a message, omitting details like whether it appends or overwrites, limits, or return behavior. The brief text does not elaborate on side effects hinted by openWorldHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that directly communicates the tool's action with no extraneous information. It is front-loaded and efficient, fitting the conciseness ideal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (two parameters, no output schema) and annotations, the description is minimally adequate. It explains the basic operation but lacks context about crowd projects, permissions, or side effects. For a write operation with openWorldHint=true, more completeness would improve agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters ('Collection ID' and 'Message content to store'). The description reinforces that the message is stored in a crowd project of a collection, but does not add new semantic meaning beyond what the schema provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Store a new message in the crowd project of a collection' clearly states the verb 'Store' and the resource 'message' in a specific context. It distinguishes from sibling tools like transkribus_crowd_delete_messages (delete) and transkribus_crowd_post_milestone (post milestone) by focusing on storing a message. However, it could be more precise about what a 'crowd project' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description does not mention prerequisites, such as requiring a collection to have a crowd project, or when not to use it. Implied usage is only for storing a message, but explicit context is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds no behavioral context beyond stating it's for a collection, which is already in the schema. Without annotations, this would score lower, but given annotations, it's adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, 5 words - very concise. However, it could be slightly more informative while remaining succinct, such as mentioning it returns detailed info about a specific collection.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description does not mention what 'details' are returned (e.g., status, settings, contributions). For a simple get tool, more context about the return value is needed to complete the picture.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear 'Collection ID' description. The description adds no additional meaning to the parameter, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get crowdsourcing details for a collection' - a clear verb and resource. However, it doesn't differentiate from sibling tools like transkribus_crowdsource_list or transkribus_crowd_get_project, leaving ambiguity about what specific details are retrieved.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use, when-not-to-use, or alternative tool suggestions are provided. The description gives no context about appropriate usage compared to sibling crowdsourcing tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false (modifies data) and destructiveHint=false (non-destructive). The description only states the action but adds no additional behavioral context such as whether the operation is reversible, what happens on error, or how the mapping is applied. The description adds minimal value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. However, it lacks structure (e.g., bullet points or examples) that could improve readability. It is efficient but minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has three parameters (one a complex object), no output schema, and no annotations indicating side effects, the description is incomplete. It does not explain the format of the body object, expected behavior when image names are missing, or any return value. An agent would need additional documentation to use this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all three parameters have descriptions). The description does not add meaning beyond the schema; 'body' is described as 'Image name mapping data' which is still vague. Baseline 3 applies because the schema already provides descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (move), resource (pages within a document), and method (based on image names). It distinguishes this tool from siblings like transkribus_page_move, which moves individual pages by ID. The verb+resource+scope is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as transkribus_page_move or transkribus_doc_get_image_names. It does not mention prerequisites, context, or scenarios where this tool is preferred. No exclusions or when-not-to-use are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate non-read-only, non-destructive, and idempotent behavior. The description confirms 'update' but adds no additional context beyond that. It does not explain what happens if the transcript doesn't exist, concurrency effects, or the extent of the update (full replace vs. partial).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, clear and direct. No redundancy. Front-loaded with the action. Could be slightly more descriptive, but overall concise and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Missing critical context: no output schema, no details on return value or error behavior. Does not explain if the update replaces the entire transcript or applies to specific fields. With 5 parameters and complexity of transcript updates, the description is insufficient for full understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all parameters. The description does not provide any additional meaning or usage tips for parameters like 'status' or 'fileName'. Baseline 3 is appropriate since schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates a transcript for a specific page in a document. It distinguishes from sibling tools like transkribus_page_post_transcript (which likely creates a new transcript) and transkribus_page_update_transcript_status (which only updates status). The scope is specific and the verb 'update' is appropriate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_page_post_transcript or transkribus_page_update_transcript_status. It does not mention prerequisites (e.g., existing transcript or page) or contextual conditions (e.g., page locking).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true, so the description adds no additional behavioral context. It does not explain side effects, permissions required, or what happens upon deletion (e.g., permanent removal, cascade effects). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words. However, it could be slightly expanded to include context without becoming verbose. Front-loaded with the key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a deletion tool with no output schema and rich sibling tools, the description lacks completeness. It does not mention that the feature must exist, the relationship to editorial declarations, or any preconditions. Minimal contextual information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with descriptions for all three parameters (collId, featId, id). The tool description adds no extra meaning beyond what the schema provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete'), the resource ('editorial declaration feature'), and the scope ('from a collection'). It distinguishes from siblings like 'transkribus_editdecl_list_features' and 'transkribus_editdecl_post_feature' by specifying deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., 'transkribus_editdecl_delete_option' for deleting options, or 'transkribus_doc_get_editorial_declaration' for viewing). The description lacks context on prerequisites or consequences.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description aligns with these (read-only count operation) but adds no additional behavioral context such as authorization needs or rate limits. With annotations carrying the burden, a score of 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise, but it omits important filter options (userid, collId, etc.), making it slightly misleading. It earns its place but could be improved by including all filter types.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (7 parameters, no output schema, but good annotations), the description is incomplete because it only mentions two filters. It does not describe the return format. A more complete description would list all filtering options and clarify the output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 7 parameters with 100% description coverage. The description names only two parameters (status, type) and does not add further meaning beyond the schema descriptions. Baseline of 3 applies per guidelines due to high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'total number of jobs'. It mentions optional filtering by status or type, but omits other available filters (e.g., userid, collId), which slightly reduces clarity. It does not differentiate from sibling tools like transkribus_job_list or transkribus_job_get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs. alternatives. For instance, there is no mention that for detailed job info one should use transkribus_job_get or for a list use transkribus_job_list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true, which cover safety and idempotence. The description adds no behavioral context beyond 'Get error details', such as whether the output is paginated or sorted. With annotations providing the core transparency, a score of 3 is appropriate as the description does not add value but does not contradict either.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the verb and resource. It is concise and to the point, but could potentially add a bit more detail without sacrificing brevity (e.g., mentioning pagination). Still, it is efficient and not verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description should ideally indicate what the tool returns (e.g., a list of error objects, error count, etc.). It does not. It also omits any mention of pagination behavior despite having pagination parameters. The annotations provide safety info but do not cover return semantics. For a tool with 5 parameters and no output schema, the description is insufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions for all 5 parameters, including an 'id' required field and optional pagination controls. The description does not add any additional meaning or context to these parameters (e.g., how sorting applies to errors). Baseline 3 is correct given full schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'error details for a specific job'. The name is explicit and the description matches. However, it does not differentiate from sibling tools like transkribus_job_get which might also return job info including errors, but the specificity is adequate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it does not mention that it should be used after a job has failed or to inspect errors, nor does it exclude any scenarios. The agent is left to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds no behavioral context beyond 'restart', such as side effects, permission requirements, or what 'restart' entails (e.g., aborting and re-running jobs).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no fluff. It is appropriately concise for a tool with one parameter, though it could include slightly more context without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool action (restarting all jobs) and lack of output schema, the description is too minimal. It does not explain what 'restart' does precisely, required permissions, or potential consequences, leaving the agent under-informed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter 'userid' described as 'User ID whose jobs to restart'. The description ('for a specific user') adds no additional meaning beyond the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('restart') and the target ('all jobs for a specific user'). The name includes 'restart_all_by_user', which differentiates it from siblings like 'job_kill' or 'job_undo' that perform different operations on jobs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., 'job_kill' or 'job_undo'). It does not mention prerequisites, scenarios, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate it's a write operation (readOnlyHint=false) but non-destructive (destructiveHint=false). The description adds no behavioral context beyond 'create', such as whether the query is saved or immediately usable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence. It is front-loaded and efficient, though it sacrifices some detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema and the description does not explain how this tool fits into the KWS workflow (e.g., relationship to searching or retrieving hits). Incomplete for a creation tool with related siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so descriptions already exist for all parameters. The description adds no extra meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('create') and the resource ('keyword spotting query'). It distinguishes from sibling KWS tools like listing or getting hits.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus siblings (e.g., transkribus_kws_list_queries, transkribus_kws_get_hits). No prerequisites or context for using the query creation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the description adds no new behavioral information. It does not contradict annotations, but adds minimal value beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, very concise and front-loaded. However, it is somewhat vague and could benefit from additional context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is incomplete for a tool with no output schema. It does not describe what the result data contains, nor does it mention any relationship to other KWS tools (e.g., requiring a prior query).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'id' has a schema description 'Resource ID', and the tool description does not add any additional context about what this ID represents. Schema coverage is 100%, so baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'result data', and the context 'keyword spotting query'. It distinguishes from sibling tools like transkribus_kws_get_hits and transkribus_kws_get_keywords by focusing on 'result data', but lacks explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not specify when to use this tool vs. other KWS tools, nor does it mention prerequisites such as needing a query to exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, which adequately convey the tool's safety profile. The description adds no behavioral details beyond a simple list operation, which is acceptable given the annotations but misses the opportunity to mention features like filtering or pagination.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, very concise and to the point. It could be slightly expanded to add context without harming conciseness, but it avoids unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 7 optional parameters and many sibling tools, the description is too minimal. It does not mention filtering, sorting, pagination, or return format, which are important for proper use. The schema provides details, but the description lacks overarching context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with each of the 7 parameters well-described in the input schema. The description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List keyword spotting queries' clearly states the verb 'List' and the resource 'keyword spotting queries', making the purpose understandable. However, it does not differentiate this tool from closely related siblings like transkribus_kws_create_query or transkribus_kws_get_hits, which also deal with KWS queries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as when to list queries versus creating new ones or retrieving hits. There are no hints about preferred contexts or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (destructiveHint: true, idempotentHint: true) already indicate the tool modifies state but is idempotent. The description does not add any additional behavioral context beyond what annotations convey, such as clarifying that pages are only unassigned from the label, not deleted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence) with no redundant information. However, it could be slightly more informative without becoming verbose, hence a 4 rather than 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no output schema, the description should clarify what the operation actually does (e.g., pages are unassigned but not deleted). It fails to provide this context, making it less complete for an agent to fully understand the tool's effect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents the two parameters (labelId, pageIds). The description adds no further meaning or usage details about these parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Remove pages from a label' clearly states the action (remove) and resource (pages from a label). It is specific enough to distinguish from sibling tools like label_remove_documents or label_remove_all_pages, though it does not explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as when to remove specific pages versus all pages. No prerequisites or contextual cues are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotentHint=true, readOnlyHint=false, and destructiveHint=false. The description adds that it operates on a 'specific page' and 'replaces' assignments, which is consistent but does not elaborate on side effects (e.g., whether previous assignments are fully overwritten or merged). Given annotations, the description provides minimal additional behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that directly conveys the core action. It is front-loaded and concise, though it could include a brief note on usage or expected behavior without adding much length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With three parameters and no output schema, the description fails to explain the semantics of 'replace' (e.g., whether labelIds overrides existing assignments or appends), the relationship between labelId and labelIds, or behavior when labelIds is omitted. This leaves the agent with insufficient information for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so each parameter (labelId, id, labelIds) is already described in the schema. The description adds no further meaning beyond what's in the schema, earning the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Replace label assignments for a specific page,' which clearly indicates the verb (replace), resource (label assignments), and scope (page). This differentiates it from sibling tools like transkribus_label_assign_pages (which assigns rather than replaces) and transkribus_label_replace_document_assignments (which operates on documents). However, it could be more explicit about the overwriting behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as transkribus_label_assign_pages, transkribus_label_remove_pages, or transkribus_label_replace_document_assignments. The agent would need to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false, destructiveHint=false, idempotentHint=false, openWorldHint=true. The description adds no behavioral context beyond these annotations. It does not mention whether the tool consumes credits, modifies the document, or creates a job. For a mutation-like tool with side effects (openWorldHint=true), more transparency is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, which is efficient. However, it could be front-loaded with more context. It is not verbose, but it sacrifices completeness for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (13 parameters, many optional) and absence of an output schema, the description is insufficient. It does not explain what the tool returns, how long it takes, or whether it is synchronous or asynchronous. A user would have to guess the behavior from the parameter names alone.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter already has a description in the schema. The tool description does not add any parameter semantics beyond what the schema provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description is specific: 'Run LA2 model inference on document pages.' It clearly identifies the action (run inference), the model (LA2), and the target (document pages). This distinguishes it from sibling tools like transkribus_la_analyze or transkribus_la_table_inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, expected input context, or cases where another tool would be more appropriate. Siblings like transkribus_la_analyze or transkribus_la_get_costs are not differentiated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (non-readonly, not destructive, not idempotent), the description adds no behavioral context such as whether the tool checks existence, returns anything, or overwrites existing associations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, no unnecessary words. Efficiently conveys the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two integer parameters and no output schema, the description is minimally complete but lacks context on side effects, error conditions, or reversibility (e.g., sibling removal tool exists).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and descriptions are provided for both parameters. The tool description does not add further meaning beyond restating parameter names, so baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add a collection to a model') and the identifiers used (model ID and collection ID). It distinguishes from removal tools but does not differentiate from the sibling 'transkribus_model_add_collection' which may have similar purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'transkribus_model_add_collection'. No prerequisites or context for usage provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, which cover safety. The description adds minimal context beyond these (e.g., requires modelId). No contradiction noted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence of 8 words with no waste. It is efficient but could be improved by briefly noting the optional filtering capability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 22 parameters and no output schema, the description is incomplete. It does not explain what 'details' includes, the filtering functionality, or how it differs from similar getter tools like transkribus_model_get_details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the baseline is 3. The description highlights the required 'modelId' parameter but adds no extra meaning for the 21 optional filter parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'details of a model by its ID', making the purpose specific. However, it does not differentiate from sibling tools like transkribus_model_get_details or transkribus_model_get_by_type, which could cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives among the many sibling tools. It implies use when you have a model ID, but no explicit context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate the tool is read-only, idempotent, and non-destructive. The description adds minimal behavioral context beyond 'filtered by type', so it relies heavily on annotations. It does not mention pagination, result limits, or performance implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence) and front-loaded with the core action. However, it could be more informative without being verbose. Still, it avoids unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 22 parameters and no output schema, the description is too minimal. It fails to explain the purpose of optional parameters, pagination behavior, or result format. The rich annotations partially compensate, but the tool's complexity demands more context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 22 parameters have descriptions in the schema, so baseline is 3. The description adds no extra meaning beyond 'optional query parameters', making it adequate but not exemplary.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves models filtered by type, which matches the tool name and distinguishes it from similar tools like transkribus_model_get or transkribus_model_list. However, it does not explicitly differentiate from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description lacks guidance on when to use this tool versus alternatives. It does not specify prerequisites or conditions for use, leaving the agent without context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare this as a read-only, idempotent, non-destructive operation, so the description does not need to restate those. However, beyond the schema, the description adds no insight into what the response contains, potential errors, or limitations. It is adequate but not helpful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded with the essential information. While it could benefit from slightly more detail, it avoids verbosity and is efficient for an agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema, the description should help the agent understand what 'field parameters' means and what to expect in the response. It does not. The tool is simple, but the description is too sparse to be fully useful on its own.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema coverage is 100%, providing descriptions for both 'id' (Resource ID) and 'type' (Model type e.g. htr, la, ocr). The description does not add additional meaning, examples, or constraints beyond the schema, so it meets the baseline with no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'field parameters for a model', specifying the required parameters 'type' and 'ID'. However, it does not differentiate from similar sibling tools like 'transkribus_model_get_details' or 'transkribus_model_get', leaving ambiguity about what 'field parameters' specifically refers to.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., other model getters). There is no mention of prerequisites, context, or contraindications. The agent must rely solely on the tool name and schema to infer usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate non-read-only and non-idempotent behavior, but the description adds no details about side effects, such as whether training can be interrupted, typical duration, or impact on existing models. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, clearly front-loaded with the action. No wasted words. Could include a bit more context without losing conciseness, but currently efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that training is a complex, long-running operation, the description lacks crucial context: it does not mention that training is asynchronous, returns a job ID, or has any callback mechanism. No output schema, so description should compensate but fails to do so.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters are described in the schema (100% coverage). The description does not add any extra semantic information beyond what the schema already provides, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('Start'), the specific resource ('P2PaLA layout analysis model training'), and the context ('for a collection'). It effectively distinguishes from sibling training tools like transkribus_pylaia_train and transkribus_recog_train_htr_citlab by specifying P2PaLA layout analysis.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., other training tools). No prerequisites, restrictions, or context for appropriate use are provided. The phrase 'for a collection' hints at the required input but does not elaborate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, indicating a safe, idempotent read operation. The description adds no behavioral details beyond the annotation flags, such as return format or pagination.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded with the action and resource. It is concise with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool and rich annotations, the description is adequate but does not explain how 'transcript text' differs from other page text tools or provide return value details. Slightly more context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with all three parameters (collId, id, page) having descriptions. The tool description does not add any parameter-specific information, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'transcript text for a specific page'. It is specific and unambiguous, but does not differentiate from similar sibling tools like transkribus_page_get_plaintext or transkribus_page_get_transcript, which may confuse selection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool over alternatives. The description does not mention prerequisites, exclusions, or context for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate a write operation (readOnlyHint=false). The description adds no further behavioral details like side effects, auth requirements, or what happens to existing transcripts, providing minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence with no redundancy. It could benefit from slightly more detail but remains efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 9 parameters, a nested object, and no output schema, the description is extremely brief. It lacks information on return values, side effects, or usage examples, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 9 parameters with descriptions, so baseline is 3. The description adds no extra meaning, failing to explain complex parameters like 'body' (what transcript data should look like) or 'status' (valid values).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Post a new transcript for a page' clearly states the action (post/create) and the resource (transcript for a page), distinguishing it from related tools like page_update_transcript or page_delete_transcript.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., transkribus_page_update_transcript for updates). No prerequisites or context provided, leaving the agent to guess.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, but the description adds no behavioral context. It does not mention that the tool modifies the document, how long it takes, or any side effects. With annotations providing minimal clarity, the description fails to disclose essential behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, appropriate for a clear purpose. However, it could be slightly more informative without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (22 parameters) and the absence of an output schema, the description is too minimal. It does not explain what the output is, how results are provided (e.g., job creation), or any post-processing needed. The description is incomplete for effective agent use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds no additional meaning beyond the schema, justifying a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Run') and the resource ('PyLaia HTR recognition on a document using a specific model'). It distinguishes this tool from sibling recognition tools (e.g., transkribus_recog_run_htr_citlab) by explicitly naming 'PyLaia' as the HTR engine.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as other HTR recognition tools or OCR tools. It lacks prerequisites, exclusions, or context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) declare safety traits, but description adds no behavioral context beyond the name (e.g., synchronous vs async, permissions, return format).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, 12 words, no waste. However, it could be slightly more front-loaded with key details like the method name, but it remains concise and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 6 parameters and no output schema, the description fails to explain what TASAS is, how to interpret parameters like ref/hyp, or what the tool returns, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all 6 parameters with descriptions, so baseline is 3. The tool description does not add any additional meaning to parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool computes Word Error Rate using the TASAS method for a document, distinguishing it from siblings like transkribus_recog_compute_wer and transkribus_recog_compute_error_rate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives (e.g., other WER or error rate methods), nor any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=false and destructiveHint=false, but description does not elaborate on side effects like job creation, resource consumption, or expected duration. Does not clarify if training is synchronous or asynchronous.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, front-loaded with verb and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, description fails to mention that the tool likely returns a job ID or status. Does not address how to check training progress or retrieve the resulting model. Lacks completeness for a training initiator tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers both parameters with basic descriptions, but the description adds no value beyond the schema. The 'config' parameter is a complex object with no structure defined; description should hint at required keys or provide example values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Start', the resource 'Layout Analysis 2 model training', and the scope 'for a collection'. It distinguishes from sibling training tools for other model types like HTR or table.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_la_la2_inference or other training tools. Does not mention prerequisites (e.g., existing training data) or that it initiates an asynchronous job.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotentHint=true and destructiveHint=false. The description merely says 'Update', which is consistent but adds no further behavioral context, such as whether the update is a full replacement or a patch, or what happens upon success. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that immediately conveys the purpose. It is concise and front-loaded with the verb and resource. However, it is minimal and could include a bit more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the body parameter (a nested object with arbitrary properties) and no output schema, the description is insufficient. It does not explain expected content for body, return value, or error scenarios. The tool is part of a larger system with many sibling tools, and the description lacks connections or prerequisites.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides descriptions for all three parameters (collId, id, body), achieving 100% coverage. The description adds no additional semantic details beyond what the schema already provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Update' and the resource 'recognition model', and specifies the context 'in a collection'. This distinguishes it from sibling tools like transkribus_recog_delete or transkribus_recog_get. However, it does not elaborate on what aspects of the model can be updated, such as name or metadata, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like transkribus_model_update or transkribus_recog_add_to_collection. It does not mention prerequisites, such as needing an existing model ID, or scenarios where other tools are more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false (modification), destructiveHint=false (not destructive), idempotentHint=true (safe to repeat), and openWorldHint=true. The description adds no behavioral context beyond this—no mention of what gets affected, permissions, or side effects. It is adequate but does not enhance transparency beyond the structured data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no unnecessary words. It is concise and to the point. However, it could be slightly improved by adding structure (e.g., listing key parameters or scope). Currently, it is efficient but minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (10 parameters, many optional) and the presence of similar siblings, the description is insufficiently complete. It does not explain return values, behavior when applied to multiple documents, or how it differs from other search-replace tools. The lack of output schema further increases the need for descriptive completeness, which is not met.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description does not add any parameter meaning beyond what the schema already provides. For example, it does not explain the difference between 'searchTerm' and 'query', or how 'docId' limits the operation. No extra detail is given.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs a search and replace within a specific collection, using the verb 'search and replace' and resource 'collection'. However, it does not differentiate from sibling tools like transkribus_search_replace, which likely operates at a different scope (e.g., document-level), missing an opportunity for distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., transkribus_search_fulltext, transkribus_search_replace, transkribus_search_keyword). It lacks 'when to use' or 'when not to use' advice, leaving the agent to infer from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, indicating a safe, read-only operation. The description adds no additional behavioral details such as pagination, response format, or filtering behavior beyond the single parameter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is appropriately front-loaded with the action and resource. However, it could benefit from a bit more context without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool with no output schema and only one optional parameter, the description is overly sparse. It does not explain what 'client files' are, how results are returned, or how the 'isRelease' filter affects the output. Given the complex ecosystem of sibling tools, this lack of context impairs correct selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters (only 'isRelease' with description and default). The description does not add any extra meaning or usage context for the parameter, so it does not compensate beyond the schema's baseline. A score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and the resource ('available client files on the server'). It effectively distinguishes from sibling tools that list other types of files or resources, such as transkribus_file_list_files or transkribus_doc_get_image_names.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There is no mention of prerequisites, limitations, or contextual scenarios. With numerous sibling tools for listing files, this omission reduces usability.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnlyHint=true, destructiveHint=false, idempotentHint=true) already indicate a safe, read-only operation. The description adds no additional behavioral information (e.g., about caching, rate limits, or error conditions). It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no wasted words. It front-loads the key action and object. It could benefit from slightly more structure, but it is appropriately brief for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description should explain what the tool returns (e.g., a version string, object with fields). It fails to do so. Additionally, it does not describe how the optional parameters affect the result, leaving gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for its two parameters ('Filter release version' and 'Package type'), so the schema already provides meaning. The description does not add any parameter context beyond what is in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'current Transkribus client version', making the tool's purpose immediately understandable. It distinguishes itself from the sibling tool 'transkribus_server_version' by specifying 'client version'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus its many siblings. There is no mention of context, prerequisites, or alternatives, leaving the agent without decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows it's a safe read operation. The description adds the concept of downloading by name but does not elaborate on what the download entails (e.g., file type, size, side effects). Given the annotation coverage, the description provides marginal additional behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded with the key action. However, it is so brief that it omits potentially important context, though it avoids verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of 3 parameters and no output schema, the description is too minimal. It does not explain what happens after download (e.g., returns binary data, a URL, or a file path), nor does it address the relationship with the sibling tool. The tool's role in the broader system is unclear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already has 100% coverage with descriptions for all three parameters (fileName, isRelease, libsMap). The description does not add any new information about parameter meaning or usage beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (download) and resource (client file) and specifies filtering by name. However, it does not differentiate from the sibling tool 'transkribus_server_download_client_file_new', missing an opportunity to clarify which tool to use for which version.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like the 'new' variant. The description lacks any context about prerequisites, expected use cases, or exclusions, leaving the agent without decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate idempotentHint=true, but the description does not mention idempotency or any other behavioral aspects (e.g., that updates are incremental or replace all). It adds minimal value beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single concise sentence that front-loads the action and resource. No unnecessary words, though slightly brief for a complex bulk operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a bulk update tool, the description lacks details on what ISAD(G) metadata entails, whether updates are additive or replacement, and what the response looks like. Given no output schema and complexity, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema fully documents the 'metadata' parameter with description. The tool description adds no additional semantic meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (bulk update) and resource (ISAD(G) metadata for uploads), distinguishing from sibling tools like transkribus_upload_bulk_update_doc_metadata and transkribus_upload_get_bulk_isad_metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor prerequisites or conditions for use. The description does not indicate scenarios where this tool is appropriate or when to avoid it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false (mutation) and destructiveHint=false (non-destructive), so the description adds little beyond confirming creation. It does not disclose side effects, required permissions, or the fact that it creates a document in the specified collection.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at only 6 words, fitting in a single sentence. However, it lacks front-loading of key details and is somewhat under-specified, but still avoids unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's role as a creation endpoint with no output schema, the description fails to explain what the tool returns (e.g., a document ID) or how it fits into the upload workflow. Sibling tools provide more context, but this description stands alone as incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides complete descriptions for all 3 parameters (collId, title, nrOfPages), so the description does not add meaningful extra information. Baseline score of 3 is appropriate given 100% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and the resource ('upload document structure'), which distinguishes it from sibling tools like transkribus_upload_create_from_mets and transkribus_upload_create_s3 by indicating a basic structure creation. However, it does not elaborate on what 'upload document structure' means, leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites or typical use cases. For example, it does not clarify that this tool is typically the first step in uploading a document, followed by adding pages.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint true, destructiveHint false, idempotentHint true, and openWorldHint true, indicating safe, idempotent reads. The description adds no further behavioral context (e.g., what metadata fields are returned, pagination behavior, or rate limits). It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—one short sentence. It is well-structured and front-loaded with the key action and resource. However, it could be slightly more informative without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having only one parameter and no output schema, the description is too minimal. It fails to explain what 'bulk metadata' entails, what the return format looks like, or how to use the tool effectively. A retrieval tool should provide more context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter collId, which has a clear description ('Collection ID'). The tool description adds no additional semantics or guidance about the parameter (e.g., optionality, default behavior when omitted). Baseline 3 applies since schema is sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Retrieve') and resource ('bulk document metadata'), and specifies the domain ('for uploads'). It distinguishes from sibling upload tools like status or update, though not explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool versus alternatives such as transkribus_upload_get_status or transkribus_upload_get_bulk_isad_metadata. The context of use is implied but not clarified.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds no additional behavioral context beyond stating the basic action. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence that conveys the core purpose without extraneous words. It is front-loaded. However, it could benefit from slightly more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With only one parameter and no output schema, the description lacks essential context about what 'bulk ISAD(G) metadata' entails and when to prefer this tool over similar ones like 'transkribus_upload_get_bulk_doc_metadata'. Absence of return value expectations or typical use cases makes it incomplete for an agent unfamiliar with Transkribus.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter (collId) with a description in the schema. The description does not add meaning beyond the schema, leaving the agent to infer that the collection ID relates to uploads. Schema coverage is 100%, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description includes a clear verb ('Retrieve') and resource ('bulk ISAD(G) metadata'), and specifies the scope ('for uploads'). However, it does not differentiate from the sibling tool 'transkribus_upload_get_bulk_doc_metadata', which suggests a similar purpose for different metadata types.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description does not mention prerequisites, exclusions, or related tools. Annotations indicate it is read-only and idempotent, but this is not leveraged in the description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotency and non-destructiveness. The description adds no additional behavioral context, such as what happens if the page already exists, or constraints on pageData.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise (one sentence) and immediately states the purpose. While it could be expanded with useful details, it avoids wordiness and is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having a nested object parameter (pageData) and no output schema, the description provides no details on how to structure pageData or what the response contains. It is too minimal for adequate understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the input schema provides meaning for both parameters. The description adds no further insight into how to use uploadId or pageData, beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Upload'), the resource ('a page'), and the target ('an existing upload'). It distinguishes from sibling tools like transkribus_page_add (which adds a page to a document) and transkribus_coll_upload_doc (which uploads a document to a collection).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool vs. alternatives. It does not specify prerequisites (e.g., the upload must exist), nor does it mention when to prefer other upload-related tools among the many siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide idempotentHint=true, destructiveHint=false, and readOnlyHint=false. The description adds the term 'initialize' but does not clarify if it creates a new user or updates an existing one, nor does it disclose side effects or authorization needs. It meets the minimum bar but adds little beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It is concise, though it may be too terse to fully inform the agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite complete schema coverage and annotations, the description omits crucial context: whether the tool creates or updates a user, what happens on duplicate ID, and the behavior when optional fields are omitted. For a tool with 5 parameters and no output schema, this is insufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%—all five parameters have descriptions. The description only references 'by ID' and does not add further meaning to parameters like username or email. Baseline 3 is appropriate as the schema already documents parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'initialize/set up' and the resource 'user account', and mentions the key parameter 'by ID'. It distinguishes from sibling tools like transkribus_user_get and transkribus_user_delete by indicating a setup operation, though the exact nature (creation vs. update) is ambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like transkribus_user_get or transkribus_user_find. There is no mention of prerequisites, scenarios, or exclusions, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, so the safety profile is covered. The description adds 'with pagination' which is already implied by parameters. No additional behavioral details like auth requirements or rate limits are provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with a single sentence. It conveys the core purpose without unnecessary words. However, it could be expanded to include more useful information without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 10 parameters and no output schema. The description does not explain the response structure, pagination semantics (how index and nValues work), or the effect of filters. It is insufficient for an agent to fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds no extra meaning beyond the schema; it only hints at pagination without clarifying parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'users', indicating the tool lists all users. It also mentions pagination, which is a key feature. However, it does not distinguish this tool from related sibling tools like transkribus_user_get or transkribus_user_find.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It lacks context on filtering, prerequisites, or when pagination is needed. No mention of use cases or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, making the behavior clear. The description adds no additional behavioral context (e.g., format, pagination, or authorization needs). It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence of 8 words that immediately conveys the tool's purpose. It is front-loaded and free of unnecessary verbiage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 string params, no output schema, comprehensive annotations), the description lacks essential details about acceptable report types, time period formats, and the structure of returned data. The agent would benefit from examples or references to documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters are documented in the input schema. The description merely echoes the parameter names ('type and time period') without adding semantic details beyond the schema's minimal descriptions. It meets the baseline but adds no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'retrieve' and resource 'administrative reports', specifying the key dimensions 'by type and time period'. It effectively differentiates from sibling tools like transkribus_admin_get_job_log or transkribus_stat_get_coll, though it does not explicitly mention alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not indicate when to use this tool over other administrative or statistics tools, nor does it mention prerequisites like admin permissions or specific report types available.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds no behavioral context beyond the name, but does not contradict annotations. With annotations, minimal extra information is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words. While minimal, it effectively states the tool's purpose. Could benefit from a bit more detail, but remains concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with three parameters and no output schema, the description is adequate but lacks explanation of what 'save activity' includes (e.g., types of saves, timestamps). Annotations cover safety, but completeness could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all three parameters (collId, from, to). The description does not add further meaning, such as the relationship between from/to or default behavior. Baseline 3 is appropriate given high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get save activity records for a collection' with a specific verb and resource. However, it does not differentiate from sibling tools like transkribus_user_activity_saves or transkribus_coll_activity_recognition, which could lead to confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Lacks context about prerequisites, such as requiring a valid collId, or when to choose this over other activity-related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already convey destructiveHint: true and idempotentHint: true, indicating it is a non-read-only but safe repeated operation. The description adds no additional behavioral context beyond what annotations provide, such as what happens if the collection is not currently a favorite or if the user is not authorized.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no redundancy. It is appropriately sized for a simple operation, but could include more contextual information without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema) and presence of annotations, the description is minimally sufficient to understand the core action. However, it lacks usage context and relationship to sibling tools, which limits its completeness for effective agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for the single parameter 'collId' (described as 'Collection ID'). The description does not add any extra meaning or constraints beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove a collection') and the context ('from the user favorites'). It uses a specific verb and resource, distinguishing it from the sibling tool 'transkribus_coll_add_favorite'. However, it could be more precise by explicitly noting that it affects the current user's favorites only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as its counterpart 'add_favorite' or other collection manipulation tools. It does not mention prerequisites (e.g., the collection must already be a favorite) or behavior if the condition is not met.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true (removal) and idempotentHint=true (safe to retry). The description adds no further behavioral context, such as whether the operation is reversible, what happens to associated data, or permission requirements. The phrase 'using the POST endpoint' is implementation detail, not behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is front-loaded with the core purpose. However, it could be slightly more informative without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple operation (2 required params, no output schema, annotations present), the description is minimally complete. It doesn't explain return value, side effects, or error conditions. For a destructive but idempotent operation, more context on what happens when the user is not in the collection would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both parameters described ('Collection ID', 'User ID to remove'). The description adds no additional meaning beyond the schema. Baseline 3 is appropriate as the schema handles parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The name 'transkribus_coll_remove_user' and title 'Remove User from Collection (POST)' clearly indicate the action (remove) and resource (user from collection). The description restates this with the same specificity. It distinguishes from siblings like 'transkribus_coll_user_add' or 'transkribus_coll_user_update_role' by focusing on removal.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool vs alternatives. For example, it doesn't mention that 'transkribus_coll_user_add_or_modify' could add or modify, or that 'transkribus_coll_user_update_role' changes roles. No mention of prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint, idempotentHint, openWorldHint. The description adds no behavioral context beyond the optional filter. For a read-only tool, this is adequate but not enhanced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, very concise. However, it omits mention of the time parameter which is present in the schema. Still, it is front-loaded and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, and the description does not clarify return format or data scope. With many credit siblings, more context would help, but for a simple read tool with rich annotations, it is minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds 'optionally filtered by type' which aligns with schema but does not mention the time parameter. Minimal added value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets credit cost information with optional type filtering. It is specific but does not distinguish from sibling tools like transkribus_credit_get which may have overlapping functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus other credit-related tools, no exclusions or alternatives mentioned. The description provides no context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, etc., covering the safety profile. The description adds only the pagination context, which is useful but does not elaborate on response format, rate limits, or data recency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, well-structured sentence that effectively communicates the core purpose without extraneous words. Front-loads the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 5 parameters, no output schema, and many sibling credit tools, the description is too sparse. It fails to explain what 'credit usage history' entails, how pagination parameters work, or what the response looks like, making it difficult for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter already has a description. The tool description adds no additional meaning beyond the schema; it only generically mentions pagination, which is already implied by index and nValues parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'get' and the resource 'credit usage history', and mentions pagination. However, among many sibling credit tools (e.g., transkribus_credit_get_transactions, transkribus_credit_list_by_user), it does not distinguish what makes 'history' unique.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like transkribus_credit_get_transactions or transkribus_credit_list_by_user. There is no mention of prerequisites, use cases, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. Description adds no further behavioral details but does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with no unnecessary words. Could be slightly more informative, but fits the simple purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only list tool with no output schema, the description is minimally complete but lacks details on return structure or credit product specifics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are already documented. The description adds no additional meaning beyond what's in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'credit products for purchase', distinguishing it from other credit tools like create or get costs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like transkribus_credit_get or transkribus_credit_get_costs, nor any prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. Description adds pagination context but does not disclose return format, rate limits, or other behaviors beyond what annotations imply. Useful but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise single sentence without filler words. Every word adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description does not explain what fields are returned. Lacks differentiation from siblings and does not cover filtering or ordering behavior beyond schema. Incomplete for a listing tool without output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter is well described. The description adds no new meaning beyond the schema, which already explains pagination parameters. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'List credit transactions with pagination', specifying verb and resource. However, it does not differentiate from sibling tools like transkribus_credit_get_history or transkribus_credit_get_package_transactions, which also list credit-related data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. No prerequisites, context, or exclusions provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, providing a clear safety profile. The description does not add any additional behavioral context beyond what annotations provide, but it does not contradict them. With annotations covering safety, the description adds no extra value, so a score of 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise and front-loaded. It contains no unnecessary words. However, it is perhaps too brief; adding a bit more context (e.g., what the test set data contains) could improve clarity without significantly increasing length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 parameters, no output schema), the description is fairly complete. However, without an output schema, the agent does not know what structure to expect from the test set data. There is also potential confusion with the sibling tool 'transkribus_recog_get_test_set', which is not addressed. The description could be more complete by hinting at the return format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both parameters ('collId' and 'id') described as 'Collection ID' and 'Document ID' respectively. The description does not add any additional semantic information beyond the schema. Since coverage is high, the baseline score of 3 is applied.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get the test set data for a document' clearly states the verb (Get) and resource (test set data) for a specific scope (document). However, it does not distinguish this tool from the sibling tool 'transkribus_recog_get_test_set', which might also retrieve test set data but for a recognition model. The purpose is clear but lacks sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. There is no mention of when to use this tool versus alternatives like 'transkribus_recog_get_test_set' or 'transkribus_doc_get_validation_data'. The description gives no context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds no additional behavioral context beyond stating it lists features.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words. Brief but not overly terse. Could be slightly more informative while staying concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with full schema coverage and annotations, the description is minimally adequate. It does not explain return format or provide example, but the tool is straightforward.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (collId) with schema description 'Collection ID'. Schema coverage is 100%, so baseline is 3. The description adds no extra meaning about the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'List', resource 'editorial declaration features', and scope 'for a collection'. It distinguishes from sibling mutation tools like transkribus_editdecl_delete_feature and transkribus_editdecl_post_feature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description does not mention when to use this or when to avoid it, nor does it reference sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate non-read-only, non-destructive, non-idempotent, and open world. The description's verb 'Store' adds minimal context. No additional behavioral traits (e.g., side effects, authorization needs) are disclosed beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single 9-word sentence with no extraneous information. It is front-loaded with the verb and resource, achieving maximum conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should explain return values or outcomes. It does not. It also misses context about prerequisites (e.g., collection must exist), relationship to editorial declarations, or distinction from features vs options. Incomplete for a create tool with siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains all parameters. The tool description adds no further meaning or constraints (e.g., uniqueness of title, relationship to collection). Baseline score of 3 applies as schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Store' and the resource 'editorial declaration feature' in a collection. However, it does not differentiate from sibling tools like transkribus_editdecl_post_option or transkribus_doc_post_editorial_declaration, so an agent may not know when to use this specific tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. No 'when', 'when not', or mention of alternative tools. The agent is left without context for selecting this tool over siblings like transkribus_editdecl_post_option.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate read-only, idempotent, non-destructive behavior. Description adds no additional behavioral context beyond stating it lists courses.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, no wasted words. However, it could be slightly more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple with optional parameters and good annotations. However, no output schema exists, so description should hint at what the list returns (e.g., course IDs/names). It does not.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 4 parameters are fully described in schema (100% coverage). Description adds no parameter-level semantics beyond what schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists e-learning courses, distinguishing it from sibling tools like count, subscribe, and unsubscribe.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_elearning_count. No when-not-to-use or contextual hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds the constraint 'completed job' beyond annotations (destructiveHint, idempotentHint). But doesn't disclose what exactly gets destroyed (e.g., transcripts, model updates) or authorization needs. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, front-loaded with action. Excellent conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimal completeness: lacks details on scope of undo, reversibility, prerequisites, or consequences. For a destructive tool, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with 'Resource ID' for jobId. Description doesn't add meaning beyond that. Baseline 3 for full coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Undo' and resource 'results of a completed job', distinguishing it from job_kill (running jobs) and job_restart. However, 'undo' is vague; does it revert to pre-job state or just remove outputs? Title 'Undo Job' slightly misaligns with 'undo results'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use vs alternatives. Implies only for completed jobs but doesn't state exclusions (e.g., running jobs). Sibling tools like job_kill and job_restart_all_by_user are not mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true. The description adds no behavioral context beyond restating the purpose. It does not disclose pagination behavior, result order, or what happens when no hits are found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no extraneous information. Every word contributes to conveying the core functionality.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema and the tool's role in a retrieval workflow, the description is too sparse. It does not explain what 'hits' represent, how to interpret results, or how this tool fits into the KWS process (e.g., after query creation).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are fully described in the schema. The description adds no additional meaning beyond what the schema provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get hits') and resource ('for a keyword spotting query'), distinguishing it from sibling tools like transkribus_kws_create_query (creates queries) and transkribus_kws_get_keywords (retrieves keywords). The purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs. alternatives. It does not mention prerequisites (e.g., a query must already exist), nor does it explain the relationship to siblings like transkribus_kws_list_queries or transkribus_kws_get_result_data.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate openWorldHint=true and idempotentHint=false, but the description adds no behavioral context. It does not explain whether assignments are additive or overwriting, or if the tool has side effects. The description fails to clarify behavior beyond the minimal verb.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that front-loads the key action and object. It contains no unnecessary words and is appropriately short for a tool with a clear name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the tool's simplicity, the description is too minimal. It lacks information about whether the operation is additive or full replacement, and does not mention potential constraints like the label must exist. Given no output schema, more context is needed for a complete understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters. The description adds no extra meaning or context to the parameters, achieving only the baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Assign documents to a label' clearly states the action (assign) and the objects (documents to a label). It distinguishes from sibling tools like transkribus_label_assign_pages (assign pages) and transkribus_label_replace_document_assignments (replace assignments), making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention that this tool adds documents to a label (as opposed to replacing assignments), nor does it reference any prerequisites such as the label existing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotent and non-destructive behavior, but the description adds no further behavioral context such as return values, permission requirements, or effect scope. The agent cannot infer key details like whether the update returns the modified label.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that directly states the purpose. It is concise and front-loaded with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having only 3 parameters and no output schema, the description omits critical details such as return value, error behavior, and whether partial updates are allowed. For a CRUD update tool, more context is needed for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter (id, name, color). The description adds no additional meaning beyond the schema, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and resource ('existing label') with the method ('by ID'). It effectively distinguishes this from sibling tools like create, delete, list, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as transkribus_label_create or transkribus_label_delete. No explicit context about usage conditions or exclusions is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description aligns with annotations (destructiveHint=true, readOnlyHint=false), confirming it is a destructive operation. However, it does not add behavioral context beyond what annotations already convey, such as irreversibility or permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no extraneous words. It is appropriately concise but could benefit from a brief example or additional context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (2 params, no output schema) and high schema coverage, the description is minimally adequate. It does not explain return values or side effects, but annotations partially compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description merely restates 'by its type and ID', adding no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('delete'), resource ('model'), and distinguishing criteria ('by its type and ID'). However, it does not explicitly differentiate from sibling deletion tools like transkribus_model_delete, which may delete by model ID alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus other model deletion tools (e.g., transkribus_model_delete, transkribus_model_remove_collection). There is no mention of prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description does not need to repeat safety traits. However, it adds minimal behavioral context beyond the annotations, such as explaining pagination via index/nValues or describing the nature of the returned validation data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the core purpose. No extraneous words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite rich annotations and full schema coverage, the description lacks explanation of the output (no output schema), pagination behavior (index/nValues parameters), and differentiation from sibling tools. It feels incomplete for a tool with 4 parameters and no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all 4 parameters having descriptions. The tool description does not add any additional meaning or context beyond what the schema already provides, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get'), the resource ('validation data'), and the key identifiers ('by type and ID'). It is specific enough to distinguish the tool from generic model retrieval, but does not explicitly differentiate from sibling tools like transkribus_model_get_validation_data_docs or transkribus_model_get_validation_data_stats.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., model_get_validation_data_docs, model_get_validation_data_stats). There is no mention of prerequisites, context, or exclusions, leaving the agent to infer usage without support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds no behavioral information beyond what the annotations provide, such as pagination behavior (index, nValues) or error cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no unnecessary words. While it is appropriate in length, it could be slightly more structured to highlight required parameters or usage context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description omits details about the return format and pagination behavior (index, nValues). Annotations cover safety, but for a query tool with no output schema, more context about what is returned would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all parameters described in the schema. The description does not add any additional meaning beyond what the schema already provides, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'documents used as validation data for a model', distinguishing it from sibling tools like transkribus_model_get_validation_data (which likely retrieves the data itself) and transkribus_model_get_validation_data_stats (which provides statistics).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no context about the model type or validation data. Usage is only implied by the tool's name and description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds 'optional pagination and filtering' but does not elaborate on default behavior, result format, or limits. It adequately confirms the read-only nature but adds little beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that captures the core functionality with no wasted words. It front-loads the purpose and main features effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 12 parameters, many sibling tools, and no output schema, the description lacks important context such as the return format, default pagination behavior, and how filters interact. It does not fully prepare the agent for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter having a description. The description briefly groups pagination and filtering but does not add significant new meaning beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list) and resource (models) along with key features (pagination, filtering). However, it does not explicitly distinguish from siblings like transkribus_model_get or transkribus_model_list_collections, and the term 'available' is somewhat ambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus the many sibling model tools. It lacks information about prerequisites, alternatives, or scenarios where this tool is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate the tool is read-only, idempotent, and non-destructive. The description adds no behavioral context beyond this, such as whether results are paginated or what the output contains. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the core purpose. No extraneous information. Slightly more detail could be added without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is functional for a simple list operation, but lacks details on the return format or scope. With no output schema and many similar siblings, more context (e.g., 'returns a list of collection metadata') would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters ('id' and 'type') are documented in the schema. The description echoes 'by type and ID' but adds no additional meaning or usage tips beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists collections for a model using type and ID. It is specific about the resource (collections) and action (list). However, it does not differentiate from the very similar sibling `transkribus_model_list_collections_by_id`, which may cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. Given many sibling tools for model operations (add, remove, list by ID), the description should indicate contexts or prerequisites, but it does not.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and readOnlyHint=false, so the behavior is partially disclosed. The description confirms removal but does not clarify side effects (e.g., whether the collection itself is deleted or just disassociated). This is adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence of 13 words, front-loading the main action and parameters without any fluff. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 100% schema coverage, the description does not explain the return value (no output schema) or the precise effect of 'remove'. For a destructive action, more context about what happens to the collection and model would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds no extra meaning beyond restating the parameter names. The schema descriptions themselves are clear, so the description does not improve understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (remove), the resource (collection from a model), and the required parameters (type, modelId, collId). However, it does not explicitly distinguish from the sibling tool 'transkribus_model_remove_collection_by_id', which may behave similarly but with different parameters.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'transkribus_model_add_collection' or 'transkribus_recog_remove_from_collection'. The description lacks context about prerequisites or situational appropriateness.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Says 'assign as a new transcript' contradicts idempotentHint=true (creating a new transcript each call is not idempotent). No other behavioral traits disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with clear action and no wasted words. Front-loaded with purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 10 parameters, no output schema, and an annotation contradiction, the description lacks important context about behavior (e.g., idempotency, relationship to existing transcripts).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so description adds no new meaning. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action (assign), resource (page), and content (plain text). Distinguishes from siblings like transkribus_page_get_plaintext and transkribus_page_post_transcript.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives (e.g., transkribus_page_post_transcript). No prerequisites or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, so the agent knows the tool is destructive. The description adds no additional behavioral context (e.g., whether page numbers shift, if transcripts are also deleted, or if the operation is irreversible). It merely restates the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence with no extraneous words. It is front-loaded with the purpose. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks context about return values (no output schema), side effects (e.g., impact on document structure), prerequisites (e.g., permissions), and error cases. For a simple delete operation, more completeness is expected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with each parameter having a description (collId, id, page). The tool description adds no extra meaning beyond the schema. Baseline 3 is appropriate as the schema carries the descriptive burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a page from a document.' uses a specific verb ('Delete') and clearly identifies the resource ('a page from a document'). It distinguishes this tool from sibling tools such as transkribus_page_add (add a page) and transkribus_page_move (move a page).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as when not to use it (e.g., for hiding a page) or any prerequisites. No mention of alternatives like transkribus_page_replace or transkribus_page_update_hide_on_sites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true, but the description adds no further behavioral context (e.g., effects of deleting a nonexistent key, side effects of openWorldHint=true). Minimal value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, clear sentence with no filler words. Front-loads the action and resource effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema and destructive nature, the description lacks information on return values, error handling, or what happens when the transcript key is invalid. OpenWorldHint=true suggests possible side effects not explained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All four parameters have descriptions in the input schema (100% coverage), so the schema already defines their meaning. The description does not add any extra semantic detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (Delete) and the resource (transcript version from a page), distinguishing it from sibling tools like transkribus_page_delete (which deletes the page) and transkribus_page_get_transcript.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites mentioned, and no exclusion criteria provided. The agent must infer usage from context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate non-readOnly (destructiveHint false, but idempotentHint true). The description adds minimal context: 'for editing' implies preventing concurrent edits. It doesn't elaborate on state change effects or conflicts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence is concise and front-loaded. However, it could be slightly improved by including quick action guidance (e.g., 'Set type to true to lock'). Still efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple boolean action tool with good annotations and schema, the description is minimally complete. It lacks mention of return value or side effects (e.g., locking prevents other edits), but the simplicity reduces need for more.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with clear descriptions for all 4 parameters. The tool description adds no extra meaning beyond what the schema provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Lock or unlock a page for editing' clearly states the action and resource. It distinguishes from sibling tools like transkribus_page_is_locked (check status) and transkribus_page_list_locks (list locks) by being an action-oriented tool, though not explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to lock vs unlock, or when to use this tool instead of alternatives like transkribus_page_is_locked. The description lacks context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description says 'replace' implying mutation, but annotations show destructiveHint=false and idempotentHint=true; no further behavioral details provided (e.g., what happens to old image, permissions needed).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, no redundant information. Every word serves a purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Missing critical context: how the new image is provided (no image parameter in schema), no output specification, and no mention of side effects. Despite simple parameters, the tool is a mutation and should explain what happens to the old image.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover 100% of parameters; description does not add new meaning beyond what the schema provides. Baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb (replace) and resource (page image), distinguishing it from sibling tools like add, delete, move.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like transkribus_page_add or transkribus_page_delete. Lacks prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate mutation (readOnlyHint=false) and non-destructiveness, but the description does not disclose that running ATR likely creates an asynchronous job, consumes credits, or what the side effects are. The openWorldHint=true is not explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no redundancy, directly conveys purpose. Efficient for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, no mention of job lifecycle (e.g., returns job ID), no explanation of how to monitor or retrieve results. For a complex, asynchronous tool, this is insufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions (100% coverage), so the description adds marginal value. It implicitly distinguishes that pageNr is for specific pages, but this is already in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Run Automatic Text Recognition') and the resource ('a document or specific pages'), distinguishing it from sibling tools like transkribus_recog_run_htr_citlab and transkribus_recog_run_ocr by specifying 'ATR'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., when to choose ATR over HTR or OCR). Missing context on prerequisites like model selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, but the description does not add meaningful behavioral context beyond that. It does not reveal whether the operation is asynchronous, what side effects occur (e.g., job creation, document modification), or if any state changes persist. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It communicates the essential function efficiently and is front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (12 parameters, no output schema, basic annotations), the description is insufficient. It does not explain what HTR recognition produces, how results are accessed, or the nature of the operation (e.g., asynchronous job). The openWorldHint suggests external impact, but no details are provided. The presence of many similar siblings demands more context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all 12 parameters, so the baseline is 3. The description does not add extra meaning or clarify relationships between parameters (e.g., modelId must be a CITlab model, format for pages). It provides no value beyond the schema, thus scoring the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Run), the specific HTR system (CITlab), the type of operation (recognition), the target (a document), and the prerequisite (a specific model). This distinguishes it from other recognition tools like transkribus_pylaia_recognize or transkribus_recog_run_ocr.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., other recognition tools, or when not to use it). There is no mention of prerequisites, context, or exclusions, leaving the agent without sufficient direction to choose correctly among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a mutation with possible side effects (openWorldHint=true). The description adds no further behavioral detail, such as that it creates a job, consumes credits, or modifies the collection. Missing context beyond what annotations already provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single short sentence, which is concise but too terse; it could be more informative without losing brevity. It conveys the core purpose but leaves out important details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 11 parameters, no output schema, and many similar siblings, the description is insufficient. It doesn't explain what TRHTR is, what the tool returns (e.g., a job ID), or how to monitor progress. An agent would likely need additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with clear parameter descriptions. The tool description itself adds no extra parameter information, but the schema covers the semantics adequately, earning the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Run'), the resource ('TRHTR recognition'), and the context ('using a specific model in a collection'). It distinguishes from sibling tools like transkribus_recog_run_htr_citlab and transkribus_recog_run_ocr by specifying the TRHTR engine.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, no prerequisites mentioned (e.g., model must be trained, collection must exist), and no explanation of complex parameters like pages, credits, or concurrency.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states 'Start... model training', which implies a mutation, consistent with readOnlyHint=false. However, it does not disclose side effects (e.g., asynchronous job, credit consumption, required permissions) that go beyond what annotations provide. Annotations are minimal, so the description should offer more behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with no unnecessary words. It is front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite being a training tool with a nested 'config' object and no output schema, the description lacks details on the training process, return value, monitoring, and typical configuration. It is insufficient for an agent to understand how to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters ('Collection ID' and 'Training configuration parameters'). The description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Start'), the specific resource ('CITlab Layout Analysis model training'), and the context ('for a collection'). It distinguishes from sibling tools like transkribus_recog_train_htr_citlab and transkribus_recog_train_la2.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as transkribus_recog_train_la2 or transkribus_la_analyze. It does not mention prerequisites, conditions, or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false and destructiveHint=false, so the agent knows this is a mutation. The description adds no behavioral context beyond what annotations provide, such as authentication requirements, side effects, or result format. Given annotations cover basic safety, a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no waste. It is front-loaded and efficiently conveys the primary purpose. However, it could be expanded slightly to include context without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple form submission tool with two string parameters and no output schema, the description is adequate but lacks post-submission behavior (e.g., confirmation, error handling). Annotations partially fill the safety profile, but the description could be more complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters ('subject' and 'description'). The description does not add any additional meaning, format expectations, or examples. Baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('submit') and resource ('bug report') with the target ('Transkribus server'). While it is specific, it does not explicitly differentiate from siblings; however, the tool name itself provides unique identification among many sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor are there any exclusions or prerequisites mentioned. The description assumes the agent knows when to submit a bug report, but with many sibling tools, this is insufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, indicating a safe read operation. The description adds no additional behavioral details (e.g., that it returns a boolean, or requires specific authentication). It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loading the purpose. It is efficient but could include more detail without being verbose. No waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature (one parameter, read-only check), the description is minimally adequate. However, it lacks information about the return value (boolean) and the context of 'current user'. Without an output schema, the agent might benefit from knowing the result format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with a basic description of jobImpl as 'Job implementation class name'. The description does not explain what values are valid or how to find the class name. Schema handles the parameter meaning, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The name and description clearly state the purpose: checking user permission for a job type. It is specific enough to distinguish from siblings like transkribus_auth_check_session which checks session validity, not job permission. However, it could be more explicit about the job type context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as transkribus_admin_authorize_users_for_job or transkribus_user_get_job_acl. The description provides no context for when a permission check is needed or what constitutes a 'job type'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds the scope 'associated with a collection' but does not disclose additional behavioral traits like pagination or sorting behavior. With annotations covering safety, a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with 7 words, highly concise and front-loaded. However, it lacks any additional context that would justify a perfect score; it is efficient but minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 6 parameters including pagination (index, nValues) and filtering (sortColumn, sortDirection, isDeleted), the description fails to mention these capabilities or any output format. No output schema exists, so more explanation was needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 6 parameters have descriptions in the schema. The tool description adds no parameter-level details beyond what is already documented. Baseline 3 is correct as the schema carries the burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'labels' with the qualifier 'associated with a collection', which distinguishes it from sibling tools like transkribus_label_list that list all labels.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., transkribus_label_list) nor any prerequisites or exclusions. It is a simple statement without usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds no behavioral context beyond annotations, but it does not contradict them. Baseline score is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words, and it is front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description could clarify the return value (e.g., boolean or object). It does not describe the output format or potential errors, leaving the agent with incomplete context for interpreting results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the 'collId' parameter described as 'Collection ID'. The description does not add further meaning beyond what the schema provides. Baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves the subscription status for a collection, using a specific verb and resource. It distinguishes itself from sibling collection tools that focus on metadata, listing, or modifications, though it does not clarify what 'subscription status' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_coll_get_metadata or transkribus_crowdsource_subscribe. No when-not or alternative tool mentions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds minimal behavioral context beyond annotations. Annotations already indicate idempotentHint=true and non-destructive, but description doesn't mention side effects like the sendMail parameter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence conveying the core purpose with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Missing details like allowed role values, permission requirements, and return behavior. Adequate for a simple update but could be more helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters have descriptions in the input schema (100% coverage). The description adds no additional parameter meaning, such as acceptable role values or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (update) and the resource (role of a user in a collection). It distinguishes from sibling tools like add/remove user, though not explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, no prerequisites or permissions mentioned. The description lacks context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds 'List', which is consistent but not additional. No extra behavioral context such as pagination behavior or auth requirements is provided. With annotations covering the safety profile, this is adequate but not enhanced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that conveys the essential purpose with no unnecessary words. Perfectly concise and structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description could mention return structure or pagination behavior. It is minimal but functional for a simple list operation. Adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so each parameter already has a clear meaning in the schema. The description does not add any semantic value beyond what the schema provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists credit transactions for a specific package. The verb 'list' and resource 'credit transactions for a specific credit package' make the purpose clear. However, it does not explicitly differentiate from sibling tools like transkribus_credit_get_transactions, though the name itself is specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There is no mention of prerequisites, exclusions, or typical scenarios. The agent must infer usage from context alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate idempotentHint=true and destructiveHint=false, which conflict with the mutation nature of 'update', but the description aligns with the annotation by not claiming destructive behavior. However, it adds no additional behavioral context beyond what annotations provide (e.g., permissions, side effects on associated jobs).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no waste. It is concise but could benefit from slightly more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (3 parameters, no output schema, annotations present), the description is minimal. It lacks context about return values, usage lifecycle, or relationships to sibling tools, leaving gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter having a basic description. The tool description adds no additional meaning beyond restating that it updates by ID. With high coverage, a score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'update', the resource 'credit package', and the identifier method 'by ID'. It differentiates from sibling tools like 'transkribus_credit_create' and 'transkribus_credit_list'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not specify when to use this tool versus alternatives (e.g., credit_create for new packages, credit_manage for broader operations) or any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true, so the behavior is partially disclosed. The description adds that messages are deleted from a crowd project, but does not elaborate on side effects, authorization needs, or irreversibility beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, concise and front-loaded. However, it is somewhat vague and missing important details, but it remains efficient without unnecessary verbiage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool with no output schema, the description is incomplete. It does not explain the effect of omitting the optional id, whether deletion is permanent, or what authorization is required. Given the tool's complexity (destructive, batch possible), more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions for both parameters (collId as 'Collection ID', id as 'Specific message ID to delete'). The description does not add meaning beyond the schema; it does not clarify, for example, that omitting id might delete all messages or that deletion is scoped to a collection.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete messages from the crowd project of a collection.' It uses a specific verb (Delete) and resource (messages), and distinguishes from siblings like transkribus_crowd_delete_milestones (which deletes milestones) and transkribus_crowd_post_message (which creates messages).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., when to delete a specific message vs. all messages, or prerequisites like collection ownership). Sibling tools such as transkribus_crowd_post_message are not mentioned, nor are usage contexts or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds 'Get all transcript IDs' but does not disclose additional behavioral traits like token usage, rate limits, or default pagination behavior. It matches annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded with the core action. It is efficient, though could be slightly expanded to include the result nature without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 7 parameters (4 optional filtering parameters) and no output schema, the description lacks information about response format, pagination, and the meaning of 'transcript IDs'. It does not explain how to use the page range, status filter, or index/nValues parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains all 7 parameters (collId, id, pages, index, nValues, status, skipPagesWithMissingStatus). The description adds no parameter-level meaning beyond 'for a document'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all transcript IDs for a document' uses a specific verb ('Get') and resource ('transcript IDs for a document'), clearly distinguishing it from sibling tools like transkribus_doc_get_fulldoc or transkribus_doc_get_page_ids.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as transkribus_doc_get_page_ids or transkribus_page_list_transcripts. No context or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds no behavioral context beyond the annotations. Annotations indicate readOnlyHint=false and destructiveHint=false, but the description does not clarify side effects, authentication requirements, or any other behavioral traits. More details could help the agent understand the tool's impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no extraneous information. Every word is necessary, making it highly concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema, no nested objects), the description is minimally adequate. However, it omits details like the fact that it applies to the authenticated user and does not describe the response or any side effects. Sibling tools exist that could benefit from more comparative context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already describes 'collId' as 'Collection ID' (100% coverage). The description mentions 'for a collection' but does not add further meaning or clarify the parameter's role beyond what the schema provides. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Subscribe') and the resource ('e-learning course for a collection'). It effectively distinguishes from sibling tools like 'transkribus_elearning_list' and 'transkribus_elearning_unsubscribe', making the tool's purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., list, unsubscribe). It lacks information about prerequisites, the context in which subscription is appropriate, or any conditions that might affect usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true, so the description carries a low burden. However, it adds no behavioral details beyond 'status or existence', leaving ambiguity about what 'status' entails or the return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that directly conveys the tool's purpose without any extraneous words. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal for a simple read-only check, but it does not specify the return value or behavior when the file is missing. Given the lack of an output schema, some additional detail would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, meaning the schema already describes both parameters (path and fileName) with clear descriptions. The tool description adds no additional meaning beyond restating that a file path is involved, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool checks the status or existence of a file at a given path, using a specific verb (check) and resource (file). It distinguishes itself from sibling tools like transkribus_file_list_files, which list files rather than check a single file.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool instead of alternatives like transkribus_file_list_files or transkribus_file_list_dirs. The description lacks context for appropriate use or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate idempotent, non-destructive, non-read-only behavior. The description 'update properties' is consistent but adds no additional behavioral context. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded and contains no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple update with three parameters and no output schema, the description is minimal but adequate. However, it lacks details on allowed status values and behavior of updates (e.g., overwrite or append).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for each parameter. The tool description does not add further semantic detail; baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates properties of an existing job. It distinguishes from sibling tools like job_create, job_get, job_list, job_kill, job_undo by using the verb 'update', but does not specify which properties beyond what is in the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like job_create or job_undo. The description lacks context for proper tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true. The description adds no behavioral context beyond the annotations, such as effects, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no fluff. It is front-loaded with the action and resource, making it easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete operation, the description is minimally adequate. However, it lacks information about return values (no output schema), error conditions, or differentiation from sibling deletion tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions. The description adds no additional meaning beyond the schema; it only references 'by ID' which matches the required id parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and the resource (label) with the identifier (by ID). It is specific and distinguishes from other label operations like create, update, list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_label_create or transkribus_label_remove_all_documents. The description does not mention prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, openWorldHint, idempotentHint, and destructiveHint. The description adds no additional behavioral context beyond restating the purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, but could benefit from slightly more detail without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Does not describe what 'detailed information' includes or the return format. Given no output schema, the description should provide more context about expected results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters (id and type). The description does not add extra meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (Get detailed information), the resource (model), and the parameters (by type and ID). It effectively distinguishes from sibling tools like transkribus_model_get and transkribus_model_get_by_type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., transkribus_model_get, transkribus_model_get_by_type). Does not mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool as read-only, non-destructive, idempotent, and open-world. The description adds no behavioral context beyond what annotations provide, such as what metadata is included or any limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 6 words, highly concise and front-loaded. Every word is necessary to convey the core purpose, with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema and the tool's simple nature, the description minimally conveys the function. However, it does not describe the kind of metadata returned (e.g., image dimensions, OCR status), which would aid completeness. Annotations partially compensate but do not cover return value details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions for each parameter (collId, id, page). The description does not add additional meaning or examples beyond the schema, so it meets the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves metadata for a specific page, using a verb ('Get') and resource ('metadata for a specific page'). It distinguishes from siblings like transkribus_page_get and transkribus_doc_get_metadata by the focus on page-level metadata, but does not explicitly differentiate among similar page-related tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as transkribus_doc_get_metadata or transkribus_coll_get_metadata. It does not mention prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, indicating a safe read operation. The description adds no further behavioral details beyond stating the check purpose, but does not contradict annotations. Without annotations, this would be insufficient, but given annotations, it is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded with the key action and resource. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Though the tool is simple, there is no output schema, and the description does not clarify the return type (e.g., boolean). It also lacks explanation of what 'locked' means in this context. However, the annotations and sibling tool names provide some implicit context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with each parameter having a description (collId, id, page). The tool description does not add any additional meaning or context for the parameters, so the baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool checks if a page is locked, using specific verb 'Check' and resource 'page'. However, it does not differentiate from sibling tools like transkribus_page_list_locks or transkribus_page_lock, which could cause confusion about when to use this specific tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as transkribus_page_list_locks or transkribus_page_lock. The description lacks context for appropriate invocation scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description provides minimal behavioral context beyond annotations; annotations already indicate idempotent and non-destructive, but description adds no new traits like permissions or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, efficiently conveys core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main functionality but lacks details on parameter constraints (e.g., negative positions) and no output schema, leaving potential gaps for complex moves.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description hints at two scenarios (within doc vs to another doc) but doesn't explain position semantics beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it moves a page within or between documents, verb+resource specific enough to distinguish from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives like transkribus_doc_move_pages_by_image_names; only describes two use cases without exclusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate that the tool is not read-only (readOnlyHint=false), not destructive (destructiveHint=false), and idempotent. The description adds the context of 'public sites', which is useful but does not elaborate on effects (e.g., whether it overrides other visibility settings). Given annotations carry the safety profile, a score of 3 reflects adequate but shallow additional information.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 8 words, extremely concise. It contains no fluff and is front-loaded with the action. While it could include a bit more context (e.g., required parameters), it earns a 4 for being efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters, no output schema, and moderate complexity, the description is too brief. It does not summarize the required inputs (collection ID, document ID, page number), mention what the tool returns (e.g., success/error), or explain the optional 'hideOnSites' parameter. This leaves the agent without essential context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 6 parameters are described in the schema (100% coverage), so the description does not need to add extra meaning. However, the description does not clarify the relationship between the two boolean parameters ('hide' and 'hideOnSites') or explain that required parameters (collId, id, page) identify the specific page. The schema does this, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Set' and the resource 'a page' with the attribute 'hidden on public sites'. This distinguishes it from sibling page update tools that deal with transcript status, locks, or other metadata.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no information about prerequisites, such as requiring specific permissions, or when not to use it (e.g., if another tool handles collection-level hiding). This lack of context makes it harder for an agent to decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate non-readonly, non-destructive, idempotent, and open-world. The description adds 'Update', confirming mutation, but provides no further behavioral details (e.g., side effects, trigger of jobs). With annotations present, a baseline of 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, extremely concise. No extra words, but slightly under-specified for its complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 7 parameters, 5 required, and no output schema, the description fails to explain status values, the note field, or the role of transcript versions in the pipeline. It is insufficient for an AI agent to fully understand tool usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 7 parameters have descriptions in the schema (100% coverage), so the description adds no new meaning. Baseline score of 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and the resource ('status of a specific transcript version'). It is specific and distinguishes from sibling tools like 'transkribus_page_get_transcript' or 'transkribus_page_delete_transcript'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'transkribus_page_update_status_v2' or other status-related tools. The description lacks context on prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, so the description does not need to repeat safety info. However, it adds no additional behavioral context (e.g., what is returned, prerequisites), which would be useful.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that directly states the tool's purpose with no unnecessary words. It is concise and efficiently structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, but with no output schema and many sibling tools, the description could provide more context (e.g., typical use case, relationship to compute_error_rate). It meets minimal adequacy but lacks comprehensiveness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 4 parameters have descriptions in the schema (100% coverage). The description does not add parameter-specific meaning beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'error rate' for a collection or model, but it does not differentiate from siblings like transkribus_recog_compute_error_rate, which may have a subtly different purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description does not clarify the distinction from compute_error_rate or other error-related tools among the many siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool as read-only, non-destructive, idempotent, and open-world. The description adds no additional behavioral context, such as what a training set includes or how it is structured.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is appropriately concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool and the presence of annotations, the description is minimally adequate. However, it lacks information about return values or what constitutes a training set, which could be helpful given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains each parameter. The description offers no additional meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get the training set for a recognition model,' which clearly specifies the verb and resource. However, it does not distinguish from sibling tools like transkribus_recog_get_train_data or transkribus_model_get_train_data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. No context about prerequisites or exclusions is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds no additional behavioral context beyond stating it searches, so it meets baseline but does not exceed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at one sentence, but given the tool has 12 parameters, it could benefit from a bit more structure or front-loading of key filters. Still, it avoids verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 12 parameters and no output schema, the description is too brief. It does not cover pagination defaults, result format, or how filters interact. More context is needed for an agent to use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters are documented structurally. The description only mentions collection and tag name, not adding new meaning beyond what the schema provides. Baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Search' and the resource 'tags', with optional filters by collection or tag name. It effectively distinguishes from sibling tools like transkribus_search_fulltext or transkribus_search_keyword.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternative search tools. The description does not specify prerequisites, exclusions, or context for optimal use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds no behavioral context beyond annotations, which already indicate destructive and idempotent nature. With annotations present, this is acceptable but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: one focused sentence with no fluff. Front-loaded with the action, easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and comprehensive annotations, the description is minimally adequate. However, it lacks context about what an 'upload' is or its lifecycle, which could be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the description adds no additional meaning beyond the schema's 'Resource ID' description. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (delete) and resource (upload) and method (by ID). It is specific and unambiguous, but does not differentiate from sibling upload tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any prerequisites or conditions such as upload status requirements. The description is purely functional.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds no behavioral details beyond restating that it gets status; it does not disclose what status information is returned (e.g., progress, error messages) or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no extraneous words. It front-loads the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one parameter and no output schema, the description is adequate but could be more helpful by indicating typical status values or response structure. Given the annotations provide safety context, the description meets minimum viability.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with 'uploadId' described as 'Resource ID'. The description merely repeats 'by ID' without adding meaning or context about how to obtain or interpret the uploadId. With high schema coverage, baseline score is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get the status of an upload by ID' uses a specific verb ('get') and resource ('status of an upload'), clearly distinguishing this tool from sibling tools like transkribus_upload_delete or transkribus_upload_create_s3.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as when to poll status versus other upload operations. No explicit when/when-not context is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the agent knows it's safe. The description adds no further behavioral traits (e.g., what is returned, pagination, rate limits). With annotations, the description is adequate but not additive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, front-loaded with the purpose. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema provided, and the description does not describe the return format of the save activity history. For a read-only tool, more detail on expected output would improve completeness. The required parameter is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. The description does not add any additional meaning beyond the schema descriptions. It does not explain that 'id' is the user's ID or that from/to define a time range.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('save activity history for a specific user'). It distinguishes from sibling tools like transkribus_coll_activity_saves by specifying 'user', but doesn't explicitly differentiate from all siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, no prerequisites, no context on when not to use it. Only a brief statement of purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds little beyond saying it gets storage usage; no additional behavioral traits like data scope or limits are disclosed. It neither contradicts nor significantly augments annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It efficiently conveys the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with good annotations, the description is adequate but lacks output details (no output schema). It doesn't mention what format the storage information returns, which could be important for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% coverage for the single 'id' parameter. The description does not add meaning beyond the schema's 'Resource ID', so it meets the baseline but doesn't compensate further.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool gets storage usage information for a specific user, with a specific verb and resource. It distinguishes from other user-related tools like transkribus_user_get (general user info) and transkribus_user_get_stats (user stats), though it doesn't explicitly differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as transkribus_stat_get_storage_usage (which might be for collections). It lacks context on prerequisites or use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotency and non-destructive behavior. The description adds no extra behavioral context beyond 'move', which is consistent but doesn't disclose details like directory creation behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that quickly conveys the core function. It is front-loaded but could be slightly more informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only 2 parameters and no output schema, the description covers the basic purpose. However, it omits details like whether the target directory must exist or if the operation is reversible.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both parameters described adequately. The description adds no additional meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Move' and the resource 'field recognition model' to a 'different directory'. It effectively distinguishes from siblings like transkribus_admin_move_table_model.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as prerequisites for the target directory or when not to move a model.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the description's claim of 'Get' is consistent. However, the description adds no extra behavior details beyond what annotations and schema imply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise and front-loaded. However, it could include more detail without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and many sibling tools, the description lacks an explanation of what constitutes 'recognition activity' or the return format. Adequate for a simple read, but incomplete for full understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are self-explanatory. Description adds no additional semantics about parameter usage or format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (Get), resource (recognition activity records), and scope (for a collection), distinguishing it from siblings like transkribus_user_activity_recognition and transkribus_coll_activity_saves.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternative activity tools such as transkribus_coll_activity_saves. The description does not provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true and idempotentHint=true, so the read-only nature is covered. The description adds no further behavioral context, such as handling of invalid collId, or whether results are filtered by default. It is adequate but does not exceed what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single clear sentence with no wasted words. It is appropriately front-loaded. However, it might be too terse for a tool with 8 parameters and no output schema; a bit more context would be helpful without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 8 parameters and no output schema, the description is minimal. It does not explain return values or indicate that filtering and sorting are available via parameters. The annotations and schema cover the rest, making it moderately complete for a simple list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the schema itself documents all 8 parameters, including collId, minBalance, includeExpired, etc. The description does not add meaning beyond 'list credit packages', so it meets the baseline but offers no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'credit packages', and specifies they are associated with a collection. This distinguishes it from sibling tools like transkribus_credit_list_by_user which list by user, and transkribus_coll_credit_history which shows history. The purpose is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as transkribus_coll_credit_history for transaction history or transkribus_credit_get for general credit info. It lacks any prerequisites, exclusions, or context about typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds no extra behavioral context (e.g., what the tool returns, whether it creates a download, any side effects). It does not contradict annotations, but the minimal description doesn't add value beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded with the key action and resource. Every word earns its place; no extraneous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description does not explain what the tool returns (e.g., a download URL, file, or status), nor does it clarify whether it exports all documents or selected ones (despite the 'pages' parameter). With no output schema, this omission significantly reduces completeness for a tool that initiates an export.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already describes all parameters (collId, pages, format) with examples and types. The tool description adds no additional meaning or clarification to the parameters, making it adequate but not helpful beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Export documents from a collection in the specified format,' providing a specific verb (export) and resource (documents from a collection). It distinguishes this tool from sibling tools like transkribus_doc_export, which exports a single document.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., transkribus_doc_export). The description lacks context for when this tool is appropriate and does not mention prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true, destructiveHint=false, and readOnlyHint=false. The description adds no further behavioral context, such as side effects of sending email (sendMail parameter) or what happens when a user already exists. The description provides minimal transparency beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, only 15 words, and immediately conveys the tool's purpose. It is front-loaded and contains no unnecessary information, making it highly efficient for an AI agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal for a tool with 5 parameters, no output schema, and many siblings. It does not mention return values, error scenarios, or optional parameters like sendMail. Given the lack of output schema and the tool's complexity, the description is insufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the parameter descriptions already exist in the schema. The tool description adds no additional meaning about parameters, such as explanation of role values or clarification of the alternative userid parameter. Baseline 3 is appropriate as the schema carries the burden.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool adds a user to a collection or modifies their role if they already exist. This distinguishes it from sibling tools like transkribus_coll_user_remove (removes user) and transkribus_coll_user_update_role (likely only updates role), as it covers both add and modify actions in one tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus its siblings, such as transkribus_coll_user_add or transkribus_coll_user_update_role. It does not mention prerequisites, when not to use it, or how to choose it over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, which cover the safety and idempotency profile. The description adds no additional behavioral context beyond the annotations, such as authentication requirements, response format, or error handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It is concise and to the point, though it could be slightly expanded to include more context without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (two parameters, no output schema, good annotations), the description is minimally adequate. However, it does not mention that the tool returns a single integer count, nor does it elaborate on the effect of the optional 'role' filter. This leaves some ambiguity for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for both parameters (collId and role), so the description does not add meaning beyond what the schema provides. The description does not explain how the 'role' parameter filters the count, but the schema already describes it as a filter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get the number of users in a collection.' This is a specific verb+resource pair that differentiates from sibling tools like transkribus_coll_user_list (lists users) and transkribus_coll_user_stats (statistics).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as transkribus_coll_user_list or transkribus_coll_user_stats. It does not mention any prerequisites, context for use, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the description's mention of optional pagination adds useful context. No behavioral traits are missing or contradictory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that directly states the tool's purpose. It is front-loaded and contains no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 6 parameters, no output schema, and siblings with similar functionality. The description only mentions pagination, omitting details on the role filter, sort options, return structure, and how pagination works. It is not sufficiently complete for an agent to use without additional context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description only vaguely mentions 'optional pagination' without connecting it to specific parameters (index, nValues). It does not add meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'list users of a collection' and mentions optional pagination, which is specific. However, it does not distinguish between this tool and the similar sibling 'transkribus_coll_user_get_list', which may cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., transkribus_coll_user_count, transkribus_coll_user_add, or transkribus_coll_user_get_list). It does not specify prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true. The description does not add further behavioral context such as side effects or permissions beyond the obvious unsubscribe action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and to the point, with no unnecessary words. It could be slightly more structured, but it is efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool, the description is minimal. It does not explain what happens after unsubscribing or any side effects, but it is arguably sufficient given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter 'collId', which is described as 'Collection ID'. The description mentions 'for a collection' but adds no new meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Unsubscribe' and the resource 'crowdsourcing project for a collection'. It is specific and distinguishes from the sibling tool 'transkribus_crowdsource_subscribe'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when one wants to unsubscribe from a crowdsourcing project, but it does not provide guidance on when not to use it or any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide clear behavioral hints (readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true). The description adds no behavioral context, but does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one concise sentence with no filler. It efficiently conveys the core function, though it could be slightly expanded without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and the richness of annotations, the description is adequate but minimal. It does not explain what METS is, which could be helpful for an unfamiliar agent. The context from schema and annotations partially compensates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for both parameters (collId and id), so the description does not need to add parameter information. It adds no additional semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'METS metadata', and the context 'for a document'. It distinguishes this tool from siblings like transkribus_doc_get_metadata and transkribus_doc_get_fulldoc by specifying the METS format.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Given the large sibling list, information about when to prefer this over other doc_get_* tools (e.g., for XML metadata retrieval) would be beneficial.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds no behavioral context beyond annotations, such as what happens if a document ID is invalid or how pages are ordered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, 11-word sentence that is direct and to the point. While concise, it could be slightly more informative without adding significant length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is provided, and the description does not explain the return format (e.g., an array of integers or strings). The tool is simple, but the lack of output specification leaves ambiguity for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters have descriptions in the schema (100% coverage). The description does not add additional meaning or constraints beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (Get), the resource (page IDs), and the scope (all pages in a document). It distinguishes from sibling tools like transkribus_doc_get_pages that retrieve full page objects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Sibling tools such as transkribus_doc_list_pages or transkribus_page_get exist, but the description does not clarify the context or trade-offs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds 'associated with a specific job' but does not reveal further behavioral traits like pagination, sorting, or result format. It is consistent with annotations but adds minimal extra context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is entirely on-topic and free of redundancy. It is concise and front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite good annotations and full schema coverage, the description lacks context about the output (no output schema) and the meaning of parameters like index, nValues, sortColumn, and sortDirection. For a tool with 5 parameters, this is insufficient for an agent to use it correctly without additional inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all parameters with descriptions (100% coverage). The description does not add any additional meaning beyond what the schema already provides. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'credit transactions associated with a specific job'. This distinguishes it from sibling tools like transkribus_coll_credit_transactions (collection-level) and transkribus_credit_get_transactions (general), making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidance is provided. The description does not mention when to use this tool versus alternatives (e.g., transkribus_credit_get_transactions), nor does it specify any prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and the description aligns by stating listing behavior. However, it adds no additional behavioral context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is front-loaded with the action (List jobs) and concisely conveys all key functionality without extraneous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description does not explain the output format or content, which is important given no output schema. It also fails to mention other relevant filters like sortColumn or userid.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description mentions pagination (index/nValues) and filters (status, type), providing minimal extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists jobs with pagination and optional filtering by status or type, distinguishing it from siblings like transkribus_job_get (specific job) and transkribus_job_count.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives; the description implies its purpose but does not provide exclusions or direct users to other tools for specific needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only states the operation name without disclosing side effects, costs, or asynchronous nature. The annotations indicate it is not read-only, but the description does not add behavioral context beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 7 words, efficiently conveying the tool's core purpose with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 11 parameters and no output schema, the description lacks information about the output format, job behavior, and implications of the parameters, leaving the agent with insufficient context for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers all 11 parameters with descriptions (100% coverage), so the description does not need to elaborate further. It merely says 'additional parameters' which is consistent with the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Run' and resource 'advanced layout analysis', and the term 'advanced' distinguishes it from the sibling tool 'transkribus_la_analyze'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies through 'advanced' and 'additional parameters' that this tool is for more complex scenarios, but it does not explicitly state when to use this tool versus the basic 'transkribus_la_analyze' or other siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=false, destructiveHint=false, and idempotentHint=true. The description adds that it updates the model, which is consistent. However, it does not disclose further behavioral traits like whether the update is partial or full replacement, or whether type is used in the URL path. Minimal added value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that clearly communicates the tool's purpose. It is front-loaded with the key verb and resource, containing no unnecessary words or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having no output schema, the description does not mention the return value (e.g., the updated model object). Given the tool's complexity and the many sibling model tools, additional context on response format or usage context would be beneficial. The description is too minimal for the domain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with each parameter having a basic description (id as 'Resource ID', body as 'Model update data', type as 'Model type (e.g. htr, la, ocr)'). The tool description does not add extra semantic meaning or clarify the structure of the 'body' object. Baseline 3 is appropriate since the schema already does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and resource ('a model'), along with the identifying parameters ('by its type and ID'). This distinguishes it from siblings like transkribus_model_update (which likely uses a different identifier) and transkribus_model_delete_by_type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not specify prerequisites, restrictions, or when not to use it (e.g., if the model type is unknown). Sibling tools exist for getting, deleting, or training models, but no differentiation is offered.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, which adequately convey safety and idempotency. The description does not add additional behavioral details (e.g., what if no transcript exists), but does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no extraneous words. Every word serves a purpose, making it efficient for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given many sibling page tools, the description fails to clarify what 'plain text' entails (e.g., no formatting, tags stripped) or behavior when no transcript is assigned. This reduces completeness for an agent to correctly select and use the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters are described in the schema with coverage of 100%, but descriptions are minimal ('Collection ID', 'Document ID', 'Page number'). The tool description adds no further semantic context beyond what the schema provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (Get) and the resource (plain text content of a page transcript). It is specific and distinguishes from siblings like 'transkribus_page_get_text' by specifying 'plain text', making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool vs alternatives such as 'transkribus_page_get_text' or 'transkribus_page_get_curr_transcript'. It does not explain scenarios or exclusions, leaving an agent to guess the appropriate context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the tool as read-only, non-destructive, and idempotent. The description adds minimal behavioral context beyond 'list all transcript versions', missing details like pagination behavior or response structure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no extraneous information. Every word is necessary and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 7 parameters, no output schema, and optional pagination/sorting parameters, the description is too brief. It fails to explain what the response contains, how pagination works, or the meaning of 'transcript versions'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so each parameter has a description. The tool description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'all transcript versions for a page', which is specific and distinguishes it from sibling tools like 'get transcript' (single version) or 'count transcripts'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as transkribus_page_get_transcript or transkribus_page_count_transcripts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds no behavioral context beyond these annotations. It does not contradict them, but also does not provide additional details like output format or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, nine words. Very concise and front-loaded. No wasted words, but could be slightly more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple read-only nature and good annotations, the description is minimally complete. However, it lacks details on the format or structure of the validation ground truth data, which could help the agent understand the output. No output schema is provided, so some additional context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters have descriptions. The description does not add any additional meaning or usage context beyond the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('Get') and resource ('validation ground truth data for a recognition model'). It is specific enough but does not explicitly differentiate from sibling tools like transkribus_model_get_validation_data, which may have similar functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives. Usage is implied (when validation GT for an HTR model is needed), but no when-not or alternative suggestions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, etc., indicating safe read. Description adds minimal context by specifying 'document transcriptions', but no further behavioral details (e.g., result format, pagination).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and scope, no unnecessary verbiage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite 7 parameters and no output schema, description only states purpose. Does not explain return value, pagination, or how to combine filters, leaving significant gaps for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with terse descriptions for all 7 parameters. The tool description does not add any extra meaning beyond the schema, so baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'Search', resource 'document transcriptions', and method 'fulltext query'. Distinguishes from sibling tools like 'search_keyword' or 'search_tags' by specifying fulltext.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like keyword or tag search. No explicit context or exclusions provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, covering safety and idempotency. The description adds no further behavioral context (e.g., download triggers network call, file size implications), but does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded with purpose. It is appropriately sized for a simple tool, though it could be slightly more informative without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 parameters, no output schema) and rich annotations, the description is minimally adequate. However, it lacks details about package types or what the download provides, which would be helpful for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with parameter descriptions like 'Download release version' and 'Package type to download'. The tool description does not add any additional meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Download the latest Transkribus GUI version' clearly states the action (download) and the resource (latest GUI version), distinguishing it from other download tools like transkribus_server_download_client_file.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as when a specific client file is needed instead of the GUI version.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true, which inform the agent that the operation is safe and repeatable. The description adds minimal behavioral context beyond identifying the resource, but does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of a single sentence with no redundant information. Every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no parameters, annotations cover safety), the description is minimally adequate. However, it lacks detail on the expected output (e.g., a health status object) that would help the agent interpret results. An output schema is absent, so extra context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With no parameters and 100% schema coverage, the baseline is 4. The description does not need to add parameter information, and it does not.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'check the status of the Transkribus image store,' specifying the verb (check) and resource (image store status). It distinguishes it from sibling tools like transkribus_system_db_status by naming a different component. However, it lacks detail on what 'status' encompasses.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., DB status, server info). The agent receives no context about scenarios where checking the image store is appropriate or when to avoid it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already convey destructiveHint=true and idempotentHint=true, so the safety profile is clear. The description adds no further context about side effects, auth requirements, or behavior on deletion of a non-existent user. It is adequate but not enhanced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with no fluff. It is appropriately sized for the tool's simplicity, though it could slightly elaborate on the effect (e.g., permanent deletion).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete operation with one parameter and no output schema, the description covers the basics. However, it lacks details on error cases (user not found) or dependencies, which would be helpful given the tool's destructive nature.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for the 'id' parameter ('Resource ID'). The description only restates that the user is deleted by ID, adding no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the target resource ('a user'), with the identifier ('by ID'). It distinguishes itself from sibling tools like transkribus_user_get and transkribus_user_list by explicitly calling out deletion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., transkribus_user_find, transkribus_user_list). There is no mention of prerequisites, permissions, or situations where deletion might fail (e.g., user in use).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint, idempotentHint. Description adds no additional behavioral context beyond what annotations provide. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, efficient, front-loaded purpose. No wasted words, but could be slightly more structured without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with clear parameters but missing return format or pagination details. Adequate but could be more complete given no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter descriptions. Description does not add meaning beyond schema, so baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'search', resource 'users', and means 'by username/email'. It distinguishes from sibling tools like transkribus_user_list and transkribus_user_get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like transkribus_user_list or transkribus_user_get. Does not mention when not to use or provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide limited info (destructiveHint=false), but 'rebuild' implies potential overwriting; description adds no behavioral traits like side effects, duration, or admin requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that is front-loaded and concise, with zero wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimal but adequate for a simple admin action. Lacks details on prerequisites, whether it triggers a background job, or impact on existing indices.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, and description correctly implies no input needed. Baseline 4 for zero-param tools, and no additional info required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb (rebuild), resource (search index), and scope (for all recognition models), providing clear purpose. It distinguishes from sibling tools like transkribus_search_reset_index by specifying models.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool, prerequisites, or alternatives. The description lacks context like 'use after model updates' or comparisons to similar tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate add operation (readOnlyHint=false, destructiveHint=false, idempotentHint=false). The description adds no further behavioral context, such as whether duplicate adds are allowed or what side effects occur (e.g., credit consumption).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, no unnecessary words. Front-loaded with the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, no output schema), the description is minimal but adequate. However, it could mention expected return value or confirmation behavior, especially since annotations are not fully informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for both parameters ('Collection ID' and 'Credit package ID to add'). The description does not add any additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Add a credit package to a collection' clearly states the action (add) and the primary resource (credit package) and target (collection). It distinguishes from sibling tools like transkribus_coll_credit_remove_package.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not indicate when to use this tool versus alternatives (e.g., transkribus_coll_credit_list for viewing packages), nor does it mention prerequisites like whether the collection or credit package must exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true, so the description adds no extra behavioral context. It doesn't mention permissions, error conditions, or the effect on related data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, concise and front-loaded. However, it could include more helpful detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple removal tool, the description covers the essential action. It is mostly complete but lacks usage context like error handling or the need to verify user membership first.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters having descriptions. The tool description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'remove', the resource 'user from a collection', and the identifier 'by their user ID'. It distinguishes from sibling tools like transkribus_coll_user_add or transkribus_coll_user_update_role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_coll_user_add_or_modify or transkribus_coll_user_update_role. No context about prerequisites or side effects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description adds that it lists collections, which is consistent but does not reveal additional behavioral traits beyond what annotations provide (e.g., potential error responses or pagination).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-formed sentence that conveys the tool's purpose without any unnecessary words or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, one parameter, and strong annotations, the description is minimally adequate. However, it lacks details about the return format (e.g., list of collection objects) and error handling, which could be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description does not add any meaning beyond the schema's 'Resource ID' description. Baseline 3 is appropriate as no extra value is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and resource ('collections') with a qualifier ('associated with a specific credit package'). It clearly states the tool's function. However, it does not explicitly differentiate from siblings like transkribus_credit_get or transkribus_credit_list_by_user, though the scope is distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when needing collections for a credit package, but provides no guidance on when to use this tool versus alternatives (e.g., transkribus_coll_credit_list) or any prerequisites (e.g., how to get the credit package ID).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide destructiveHint=true, idempotentHint=true, and readOnlyHint=false. The description states 'delete,' which aligns with destructiveness but adds no extra behavioral context (e.g., what happens if option is in use, or if deletion is permanent). With annotations present, the description adds minimal additional value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It is front-loaded with the key action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is very brief for a delete operation with no output schema. It lacks context about the editorial declaration system, relationships between options and features, or error conditions. However, annotations and schema cover basic requirements, making it adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all three parameters are described in the schema. The description does not add meaning beyond the schema, such as clarifying the difference between optionId and id or providing usage context. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'delete,' the resource 'option,' and the context 'editorial declaration feature.' It distinguishes itself from sibling tools like transkribus_editdecl_delete_feature and transkribus_editdecl_post_option, providing specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, lacks prerequisites, and does not mention when not to use it. Among sibling tools, there is no explicit comparison or context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true. The description adds no further behavioral context, such as what exactly happens upon unsubscription or any side effects. It is not contradictory, but the minimal description does not enhance transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that precisely conveys the tool's function with no unnecessary words or repetition. It is appropriately front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema), the description is minimally adequate but lacks details like required permissions or reversibility. The annotations partially compensate, but the description could provide more context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides full coverage (100%) with a clear description for the sole parameter 'collId'. The tool description adds no additional semantic information beyond what the schema already offers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (unsubscribe) and resource (e-learning course for a collection). It effectively distinguishes the tool from sibling tools like transkribus_elearning_subscribe and transkribus_elearning_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description does not mention prerequisites (e.g., being subscribed first) or contrast with related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare safe read-only, non-destructive, idempotent behavior; description adds no new behavioral context beyond listing files.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded, no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple list operation but lacks details on default path behavior and return format; no output schema to compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema describes the 'path' parameter with 100% coverage; description merely references 'given path' without adding new meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'files at a given path', distinguishing it from similar tools like 'transkribus_file_list_dirs'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives; no mention of optional path or default behavior.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds no additional behavioral context beyond what annotations provide, but does not contradict them. With rich annotations, the description's contribution is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that conveys the core purpose without unnecessary words. It is appropriately front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a single parameter and no output schema. Annotations cover safety. The description is minimal but sufficient to indicate the basic function. However, it does not explain what the returned keywords look like or provide context about keyword spotting queries, which could be helpful for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter 'id' with description 'Resource ID'. The description does not add any additional meaning beyond what the schema provides. Schema coverage is 100%, so the baseline is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'keywords for a keyword spotting query'. It is specific and distinguishes the tool from sibling tools like transkribus_kws_create_query or transkribus_kws_get_hits.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any context about prerequisites or use cases. The description is minimal and provides no exclusions or when-not-to-use information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, which adequately describe the tool's behavior. The description adds no further behavioral context (e.g., what happens if the label is not found) beyond what is implied by the name and annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. While efficient, it is extremely minimal and could benefit from additional context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with one parameter and rich annotations, the description is sufficient. It tells the agent that providing an ID retrieves the label. The lack of output schema is acceptable, as the return is likely the label object itself.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description does not add meaning beyond the schema. The schema describes 'id' as 'Resource ID', and the description simply says 'by ID', which is already clear from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a label by ID' clearly states the action (get), resource (label), and method (by ID). It distinguishes from sibling tools like transkribus_label_list (lists labels) and transkribus_label_create (creates labels).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. For example, it does not mention that to list all labels one should use transkribus_label_list, or that this tool is for retrieving a specific label when the ID is known.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. The description adds no additional behavioral context such as authorization requirements or rate limits, but does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that efficiently communicates the purpose. It is concise without being overly minimal, though it could include a bit more detail without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the rich annotations and complete schema, the description adequately covers the tool's purpose. It does not explain return values, but for a cost estimate tool, the purpose is clear enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all parameters. The description does not add any meaning beyond what the schema already provides, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'cost estimate', and the specific context 'layout analysis on a document'. This distinguishes it from similar cost tools like transkribus_credit_get_costs and transkribus_recog_get_costs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There are multiple cost-related tools, and no exclusions or context are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true, readOnlyHint=false, and idempotentHint=true. The description adds no further behavioral context (e.g., irreversibility, effects on related data). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no redundant words. Every part contributes to understanding the action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While annotations provide safety information, the description lacks details on return values, error handling, or effect persistence. Given the tool's destructive nature and no output schema, more context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions ('Model/HTR ID', 'Collection ID') that already convey the purpose. The description does not add additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly uses the verb 'Remove' and specifies the resource 'a collection from a model', with the parameters model ID and collection ID. This distinguishes it from sibling tools like transkribus_model_add_collection and transkribus_model_remove_collection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives such as transkribus_model_remove_collection. No prerequisites, disclaimers, or when-not-to-use advice is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint, destructiveHint, idempotentHint, and openWorldHint, covering the safety profile. The description adds no extra behavioral details (e.g., return type or side effects), but the existing annotations moderate the need for more transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single, efficient sentence with no wasted words. It is appropriately concise for a simple count operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the tool is simple and annotations provide good context, the description does not specify the return type (e.g., integer count) or any additional details that might be helpful. It is adequate but leaves minor ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema covers all parameters (collId, id, page) with descriptions, achieving 100% coverage. The description adds no additional meaning beyond the schema, so it meets the baseline without enrichment.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'Get' and the resource 'the number of transcripts for a page', which is specific and distinct from sibling tools like transkribus_page_list_transcripts (lists transcripts) and transkribus_page_get_transcript (gets a specific transcript).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as transkribus_page_list_transcripts or transkribus_page_get_transcript. The description does not provide context for when a count is appropriate over listing or retrieving a transcript.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds no behavioral context beyond the annotations, such as pagination behavior or response format. It is adequate but does not enhance the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded with the main action. However, for a tool with 6 parameters and no output schema, slightly more detail could be beneficial without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal and does not explain pagination, sorting, filtering parameters, or the return format. Given the tool has 6 parameters and no output schema, the description should provide more context for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with descriptions for all 6 parameters. The description does not add meaning beyond identifying the main resource (collection). Baseline 3 is appropriate as the schema already documents parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and the resource 'recognition models' with the scope 'available in a specific collection'. It distinguishes from sibling tools like transkribus_recog_list_models (which likely lists all models) and transkribus_model_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (to list models in a collection) but does not explicitly provide guidance on when not to use, nor does it mention alternatives or specify prerequisites. Usage context is implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, destructiveHint, idempotentHint, and openWorldHint, so the agent knows it's a safe read operation. The description adds that the tool returns collections with access to a model, but does not disclose details like pagination, permissions, or response format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that conveys the tool's purpose without any unnecessary words. Perfectly concise for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with only 2 parameters and no output schema, the description is adequate but lacks details about the return structure or any constraints. An agent might need to infer the output format from context or experience.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes both parameters (collId and id). The description does not add any new meaning or context about these parameters; it only repeats the overall purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('list') and the resource ('collections that have access to a recognition model'). It is specific and distinguishes from sibling tools like transkribus_recog_list_by_collection which lists models for a collection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool vs alternatives such as transkribus_model_list_collections or transkribus_recog_list_by_collection. No exclusions or context are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint and idempotentHint. The description adds no further behavioral context, such as whether the operation is reversible or cascading. It meets the minimum but does not enhance understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loaded with the key action, and contains no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool, the description is minimal. It does not clarify important details like the effect on the model itself (only removed from collection, not deleted) or prerequisites (e.g., model must exist in collection). Given the annotations and simple input, it is adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds no additional meaning beyond the schema, so it scores the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Remove', the resource 'recognition model', and the target 'from a collection'. It is specific and distinguishes from sibling tools like transkribus_model_remove_collection or transkribus_recog_add_to_collection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as transkribus_recog_delete (which might delete the model entirely) or transkribus_model_remove_collection. The description lacks context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds no additional behavioral context beyond stating the action. No contradictions, but it misses opportunities to explain what 'version' means or the expected format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no redundancy. Every word serves a purpose, demonstrating excellent conciseness and front-loading of key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters, rich annotations, and a simple action, the description is functional but minimal. It lacks details on the response format (e.g., version string structure) which could be useful for an agent, but is adequate for a straightforward version retrieval.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has no parameters with 100% coverage. The description does not need to add parameter details, but it also does not clarify return values which could be considered part of semantic context. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get the current Transkribus server version' uses a specific verb (get) and resource (server version) with clear scope. It distinguishes well from sibling tools like transkribus_server_client_version which gets client version.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or alternative guidance is provided. The simple nature implies usage when server version is needed, but no exclusions or comparisons to siblings are given, leaving the agent to infer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description is consistent with the annotations (readOnlyHint=true, destructiveHint=false), but it adds no additional behavioral context beyond what is already provided by the annotations. The tool is simple, so this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that front-loads the purpose. Every word is necessary and contributes to clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is minimal but adequate for a simple tool with no parameters and no output schema. It lacks details about the return value or what 'status' entails, which could improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema is empty with 100% coverage, so there are no parameters to describe. The description does not add parameter information, meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Check' and the resource 'Transkribus database status', making the purpose clear. However, it does not differentiate from sibling tools, and 'status' could be more specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for checking database status, but it provides no explicit guidance on when to use this tool versus alternatives, nor any exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not add any behavioral context beyond what annotations already provide (readOnlyHint, destructiveHint, idempotentHint, openWorldHint). It omits details like authentication requirements, rate limits, or return format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no wasted words. It is front-loaded with the action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, zero-parameter read-only tool, the description provides the essential information. However, it could be more complete by hinting at what specific statistics are returned, especially since there is no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters, the baseline score is 4. The description correctly implies no input is needed, and the schema is fully covered.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('usage statistics for the current user'). It distinguishes itself from sibling tools like transkribus_coll_user_stats or transkribus_stat_get_coll by specifying 'current user'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. Given the large number of sibling tools, some context on when to prefer this over other stats tools would be helpful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotentHint=true and readOnlyHint=false, which the description does not contradict. However, the description adds no additional behavioral context beyond 'create or update', missing details on overwrite behavior or limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single succinct sentence that front-loads the key action and scope. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one parameter and annotations, the description covers the basic operation but lacks usage guidance and behavioral depth. It is minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter tagDefs, so the baseline is 3. The description does not add extra meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the verb (create or update), resource (tag definitions), and scope (for the current user). It distinguishes from sibling tools like transkribus_coll_update_tag_defs which target collection-level tags.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide guidance on when to use this tool versus alternatives like transkribus_coll_update_tag_defs or transkribus_coll_update_default_tag_defs. No explicit context for selection is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotent and non-destructive behavior. The description adds no additional behavioral traits, such as the effect on model references or if moving deletes the original. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence of 11 words with no redundancy. It conveys the essential information without unnecessary detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple move operation with two parameters and good annotations, the description provides the core action. However, it does not specify whether the path is relative or absolute, or error handling for non-existent models, which would enhance completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Both parameters have clear descriptions in the schema ('Table model name to move' and 'Target directory path'). The description adds no extra meaning beyond what the schema provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action (move), resource (table recognition model), and destination (different directory). It effectively distinguishes from the sibling tool 'transkribus_admin_move_field_model' by specifying the model type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives, or prerequisites. The description implies usage when a table model needs moving, but lacks context like when not to use it or comparison to similar move tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds no additional behavioral traits beyond confirming it retrieves details by ID. With annotations present, the description is adequate but not enriched.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no redundant information. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter, no output schema, and robust annotations, the description is sufficient to explain core function. However, it could include a note about return type or usage context to be fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'id' is fully documented in the schema with description 'Resource ID'. The description only restates 'by ID' without adding new meaning. Schema coverage is 100%, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'credit package', and the required identifier 'by ID'. It effectively distinguishes from siblings like transkribus_credit_list_by_user or transkribus_credit_get_history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus other credit-related tools (e.g., transkribus_credit_list_by_user, transkribus_credit_get_history). The description lacks context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true, so the description need not repeat these. It does not add further behavioral context beyond what annotations provide. No contradiction found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that efficiently communicates the tool's purpose without extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two parameters, no output schema, and annotations providing destructive/idempotent hints, the description is adequate. It could mention that deleting without id removes all milestones, but overall it is complete enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description does not add additional meaning beyond the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete'), the resource ('milestones'), and the context ('from the crowd project of a collection'). It effectively differentiates from sibling tools like transkribus_crowd_post_milestone (create) and transkribus_crowd_get_project (retrieve).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as deleting all milestones vs. a specific one, or prerequisites like requiring milestone IDs. The description lacks contextual usage advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. Description adds no behavioral context beyond 'list directories at a given path.' No details on behavior for invalid paths, recursion, or result format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no fluff, front-loaded with action and resource. Could be expanded with more context but is appropriately concise for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple with one parameter and no output schema. Description is adequate but lacks details on return format, scope (immediate or recursive), or behavior for invalid paths. Annotations cover safety.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and schema already describes parameter 'path' as 'Directory path to list subdirectories from.' Description only restates 'at a given path,' adding no new meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool lists directories at a given path. The verb 'list' and resource 'directories' are specific, and it distinguishes from sibling transkribus_file_list_files which lists files.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage: provide a path to list directories. However, no explicit when-to-use, when-not-to-use, or alternatives among siblings. Context from sibling names helps but is not provided in description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint: true and idempotentHint: true. The description adds the scope 'all documents,' which is helpful but does not elaborate on the nature of removal (e.g., permanent unassignment vs deletion). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no unnecessary words. It conveys the core action efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with full schema coverage and no output schema, the description is nearly complete. It lacks a note about destructiveness or reversibility, but the annotations partially compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single parameter 'labelId' described as 'Label ID.' The tool description does not add any extra meaning or constraints beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Remove all documents') and the target resource ('from a label'). It is specific and effectively distinguishes this tool from siblings like transkribus_label_remove_documents (removes specific documents) and transkribus_label_remove_all_pages (removes all pages).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, no prerequisites, and no exclusions. It does not mention that the operation is destructive or irreversible, leaving the agent without critical context for decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already report destructiveHint=true and idempotentHint=true. The description adds no extra behavioral details beyond stating the delete action, but it does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, clear sentence efficiently communicates the tool's function with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter and comprehensive annotations, the description is adequately complete. It could mention permanence or prerequisites, but the basic functionality is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'modelId' is fully described in the schema as 'Model/HTR ID'. The description adds no additional meaning, meeting the baseline for 100% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete' and the resource 'model by its ID', making it specific and distinct from sibling tools like 'transkribus_model_delete_by_type'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like deleting by type. The description is minimal and does not provide context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint, idempotentHint, so behavioral safety is clear. The description adds minimal context (listing locks) beyond purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with verb and object, no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only list tool with no output schema, the description is adequate. Annotations cover behavior, but minor details like response format could enhance completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions for collId, id, and page are clear and complete. With 100% schema coverage, the description adds no further parameter detail, meeting baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all locks on a page, distinguishing it from siblings like transkribus_page_is_locked and transkribus_page_lock.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_page_is_locked for a boolean check or transkribus_page_lock for acquiring locks.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: true. The description adds minimal behavioral context beyond 'Get details', which is consistent. No additional traits (e.g., required permissions, response format) are disclosed, but the annotations cover the safety profile adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence (8 words) with no redundant information. It efficiently communicates the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only retrieval tool with two well-documented parameters and no output schema, the description is sufficient. It could hint at what 'details' include (e.g., metadata, training status), but the openWorldHint annotation reduces the need for exhaustive enumeration. Overall, it meets the minimum viable completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema provides clear descriptions for both parameters (collId: 'Collection ID', id: 'Resource ID') with 100% coverage. The description does not add further meaning beyond what the schema already conveys, so baseline score applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get', the resource 'details of a recognition model', and the scope 'in a collection'. It effectively distinguishes from sibling tools like transkribus_recog_delete (delete) or transkribus_recog_list_by_collection (list).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., transkribus_recog_list_models to list all models, or transkribus_recog_get_atr for specific model data). No when-to-use or when-not-to-use context is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so the description aligns but adds no extra behavioral context beyond stating it's a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no unnecessary words, fully front-loaded with essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with good annotations and schema, the description is mostly complete but lacks details on the output format since there is no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are documented. The description does not add additional meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'Get', resource 'recognition activity history', and scope 'for a specific user'. It differentiates from sibling tools like transkribus_coll_activity_recognition and transkribus_user_activity_saves.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. No context about intended scenarios or exclusions provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, destructiveHint, idempotentHint, and openWorldHint. The description does not add behavioral context beyond what annotations convey. It is not contradictory, but adds no extra value here.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, 8 words, front-loaded with the verb 'Get'. Every word is necessary and there is no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-ID tool with strong annotations and a single well-described parameter, the description is sufficient. It covers the purpose and required input. No output schema exists, but that is acceptable given the simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (parameter 'id' has description 'Resource ID'). The tool description says 'by ID' which essentially restates the schema. No additional meaning provided beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', resource 'details of a specific user', and the means 'by ID'. It effectively distinguishes from sibling tools like transkribus_user_list and transkribus_user_find.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like transkribus_user_list (for listing all users) or transkribus_user_find (for searching by criteria). The description implies you need a specific ID but does not explicitly state the context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true, covering the safety and idempotency. The description adds 'optionally filtered' but does not elaborate on return format or behavior beyond what annotations provide. With strong annotations, the description's added value is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, complete sentence that front-loads the purpose. Every word earns its place with no extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description does not specify the return format (e.g., a single number). With 12 optional parameters and no guidance on their interaction, the description is incomplete for an agent to fully understand the tool's behavior without additional inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage with descriptions for all 12 parameters. The description only highlights 'collection or type' filters, not adding meaning beyond schema. Baseline is 3 for high schema coverage; description does not compensate for low coverage (which is not the case).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'total count of actions', directly specifying the tool's function. It distinguishes from sibling tools like transkribus_action_list (returns list) and transkribus_action_info (returns info) by focusing on a count.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions optional filtering but provides no explicit guidance on when to use this tool versus alternatives. It does not indicate when not to use it or direct the agent to other tools for more detailed queries, leaving the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true. The description does not add any behavioral context beyond the annotations, but it does not contradict them either.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no unnecessary words. It is front-loaded with the action verb and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema, the description does not elaborate on what 'detailed information' includes. While annotations provide safety and idempotency hints, the description could be more complete by listing typical response fields. However, it is adequate for a simple retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter 'id', which is described in the schema as 'Resource ID'. The description does not add any further meaning or constraints to the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Retrieve' and the resource 'detailed information about a specific action'. It distinguishes from sibling tools like transkribus_action_list (which lists actions) and transkribus_action_count (which counts actions) by specifying 'specific action' via an ID.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you need details of a single action, but does not provide explicit when-to-use or when-not-to-use guidance. No prerequisites or alternatives are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, etc. Description adds 'crowd project details' context but no further behavioral insights. No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no unnecessary words, front-loaded with action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple read-only tool with one parameter; description is fully adequate. No output schema needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter clearly described as 'Collection ID'. Description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (get) and resource (crowd project details for a collection), distinguishing it from sibling tools like transkribus_crowd_post_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., when to retrieve project details before posting updates).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint, idempotentHint, and openWorldHint, covering safety and idempotency. The description adds no extra behavioral details (e.g., rate limits, return format). It does not contradict annotations, but also does not add value beyond them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence (8 words) that front-loads the verb and resource. Every word is necessary, and there is no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
As a simple count tool with no parameters and no output schema, the description is minimal but lacks return type information (e.g., integer). While annotations cover safety, the description would benefit from stating what the output represents (e.g., 'Returns an integer count').
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, and schema description coverage is 100%. With no parameters to document, the baseline per rubric is 4. The description appropriately avoids adding unnecessary detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get the total count of crowdsourcing projects' uses a specific verb ('Get') and resource ('total count of crowdsourcing projects'). It clearly distinguishes itself from sibling tools like transkribus_crowdsource_list and transkribus_crowdsource_get_details, which serve different purposes (listing or fetching details).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention scenarios where a count is preferred over a list, nor does it exclude any contexts. Without any usage direction, agents may misuse or overlook the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true, openWorldHint=true. The description adds no additional behavioral context beyond what annotations provide, such as pagination or result format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single sentence, concise and direct. While it could include a bit more detail, it avoids unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with comprehensive annotations and fully described parameters, the description is adequate. It clearly states what it does, though it does not mention the return format or any result limits.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% description coverage for all 4 parameters. The description does not add any extra meaning beyond what is already in the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action ('List') and resource ('crowdsourcing projects'). It is specific and distinguishes from sibling tools like transkribus_crowdsource_get_details or transkribus_crowdsource_subscribe.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. Usage is implied (when you need a list), but no exclusions or alternative tool mentions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide destructiveHint=true and idempotentHint=true. The description adds context that the operation removes without deleting, clarifying the scope of destructiveness. This is helpful beyond what annotations convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no filler. The description is maximally concise while conveying the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple operation, the description adequately states the core action. However, it omits any mention of return values, permissions, or side effects (e.g., document remains in other collections). Given no output schema, some minimal guidance would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage for both parameters (collId and id). The description does not add extra meaning beyond the schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Remove a document from a collection without deleting it.' It specifies the verb 'Remove' and the resources 'document' and 'collection,' and distinguishes from deletion. It differentiates from siblings like transkribus_doc_delete by explicitly stating the document is not deleted.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool over alternatives like transkribus_coll_remove_doc or transkribus_doc_delete. No mention of prerequisites, when to choose this over other removal operations, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnlyHint, idempotentHint, etc.) already indicate safety. The description adds that it counts only the current user's documents, but does not explain the parameters 'index' and 'nValues' which may affect behavior. Open world hint is true but not elaborated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that is front-loaded with the core purpose. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple count tool with no output schema, the description is adequate but incomplete. It does not mention the return format (likely a number) or the role of pagination parameters. Annotations fill some gaps, but behavioral context could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description does not add any additional meaning beyond the schema for parameters like 'index' and 'nValues'. The parameter 'getAllDocsIfAdmin' is also not explained in context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'total number of documents owned by the current user'. It specifies the scope (owned by current user) and distinguishes from the sibling 'transkribus_user_list_my_docs' which lists documents.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when/when-not or alternative tools are mentioned. The description implies use for counting instead of listing, but does not provide explicit guidance or exclusions. Sibling tools like 'transkribus_coll_count_docs' exist but are not referenced.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, destructiveHint, idempotentHint, openWorldHint. The description adds 'with pagination', confirming pagination support but no other behavioral details beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no unnecessary words, front-loaded with action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 11 parameters and no output schema, the description only covers pagination. It omits filtering capabilities (collId, filter, isDeleted, etc.) and sorting, which are key features of the tool. Could be more complete given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. Description only mentions pagination, which ties to index/nValues, but adds no meaning for other parameters (sortColumn, collId, filter, etc.) beyond what schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description specifies verb 'list', resource 'documents owned by the current user', and feature 'pagination', clearly distinguishing from siblings like transkribus_coll_list_docs and transkribus_user_count_my_docs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives. While the scope is implied by 'owned by the current user', it does not mention when not to use or compare with similar tools like transkribus_coll_list_docs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, destructiveHint=false, idempotentHint=true, and openWorldHint=true, fully covering the safety profile. The description adds no additional behavioral context, such as authentication requirements, rate limits, or return format details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that directly states the tool's purpose. No unnecessary words, and the key information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with no parameters and thorough annotations, the description is adequate. It clearly states what the tool does. While it could mention that it returns a list of dictionary names, the purpose is sufficiently covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema coverage is 100% (trivially). With 0 parameters, the baseline is 4, and the description does not need to compensate for parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'list' and clearly identifies the resource 'available dictionaries for recognition'. It effectively distinguishes from sibling tools like transkribus_recog_list_models, which lists models, not dictionaries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you need a list of dictionaries, but it does not provide explicit guidance on when to use this tool versus alternatives or mention any specific context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. The description adds 'for the current user' scope but no additional behavioral traits beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence of 11 words, efficient and front-loaded. No redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema or parameters, the description is adequate for a simple getter. Could optionally explain what an ACL contains, but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, schema coverage 100%. Baseline 4 applies; description does not need to add parameter info.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action 'Get', the resource 'job access control list', and the scope 'for the current user'. It is distinct from sibling tools like admin-oriented ACL tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., admin ACL tools). No mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and idempotentHint=true. The description adds no further behavioral context beyond 'list all', but given the annotations, it is not misleading. A 3 is appropriate as the description provides minimal added value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 7 words, conveying all necessary information without any waste. It is appropriately concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless tool with good annotations, the description is largely complete. However, it could mention the return format or pagination, especially since there is no output schema. Still, it sufficiently describes the tool's function.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and the schema description coverage is 100% (empty). The description does not need to add parameter details, so a baseline of 4 is justified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list), resource (tag definitions), and scope (current user). This distinguishes it from sibling tools like transkribus_coll_list_tag_defs which list per collection, and transkribus_user_update_tag_defs which updates.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, it does not mention when to use user-level vs collection-level tag listing or any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive. Description adds context that it lists authorized users for a specific job implementation, reinforcing safe behavior without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words. Could be slightly more detailed but adequate for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has one parameter, no output schema. Description sufficiently covers purpose and required input. Complete enough for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage with description for jobImpl. Description adds no extra meaning beyond schema. Baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states verb 'List', resource 'users authorized', and context 'for a specific job implementation'. It distinguishes from siblings like transkribus_admin_authorize_users_for_job.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage (when needing authorized users for a job implementation) but does not explicitly state when to use or avoid, nor mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and idempotentHint=true, so the description adds some context ('active') but does not go beyond what annotations imply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no extraneous words, achieving maximum conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the purpose is clear, the description lacks details on the output format or any prerequisites, but given the tool's simplicity and rich annotations, it is minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, and schema coverage is 100%, so description is not required to add parameter info; baseline 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists all currently active user sessions, which is specific and distinct from sibling admin tools like transkribus_admin_authorize_users_for_job or transkribus_admin_get_job_log.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide explicit guidance on when to use this tool versus alternatives, but its purpose is straightforward for listing active sessions, so usage is implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, idempotentHint=true. Description adds no further behavioral context, but is adequate given the safety profile and no parameters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single concise sentence that front-loads the action and resource. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple count operation with no parameters and strong annotations, the description is complete. No output schema is documented, but a simple count typically returns a number, which is implied.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist; schema coverage is 100%. Description adds no parameter info, but baseline is high due to no parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb 'Get' and resource 'total count of e-learning courses'. Distinct from sibling tools like transkribus_elearning_list which returns list details, and subscribe/unsubscribe.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use versus alternatives. Usage is implied: use when only the count is needed instead of the full list from transkribus_elearning_list.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true. The description adds no additional behavioral context such as side effects, return values, or what happens if the job is already stopped. It does not contradict annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one short sentence with no extraneous information. Perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the description covers the core action. However, for a destructive action, information about the return value or confirmation of success would be helpful. Lacking that, it is mostly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter 'id' is described in the schema as 'Resource ID'. The description's 'by ID' adds no new meaning. With 100% schema coverage, baseline is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Kill a running job by ID.' clearly specifies the verb (kill), resource (job), and the condition (running) and parameter (ID). It distinguishes well from sibling tools like transkribus_job_create or transkribus_job_get.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for running jobs, but does not explicitly state when to avoid using it or alternatives. However, the context of sibling tools makes the usage context clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true and idempotentHint=true, so the description does not need to repeat safety properties. However, it adds no extra behavioral context beyond stating it retrieves info. With strong annotations, a 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no extraneous information, perfectly concise for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and a straightforward purpose (listing available models), the description is sufficiently complete. It could mention the return format but is adequate for selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description is not required to explain parameter semantics. Per instructions, 0 params baseline is 4. The description correctly identifies the resource.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'retrieve' and the resource 'information about available ATR models', distinguishing it from sibling tools that operate on specific models or run recognition tasks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like transkribus_recog_list_models or transkribus_recog_get. While the purpose is clear, the description does not help select among similar tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations include destructiveHint: true and idempotentHint: true. The description adds the conditional behavior (only if empty), which is not captured by annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single short sentence that front-loads the key behavior. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple conditional delete with one parameter and no output schema, the description is adequate. It could mention error handling (e.g., if collection not empty or not found), but the core behavior is clear.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter 'collId' with schema description 'Collection ID'. Schema coverage is 100%, so description does not add extra meaning beyond the schema. Baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The name 'Delete Empty Collection' and description 'Delete a collection only if it is empty.' clearly state the action and condition. This distinguishes it from 'transkribus_coll_delete' which presumably deletes any collection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use (safe deletion of empty collections). It does not explicitly state prerequisites or when not to use, but the sibling tool name suggests an alternative for non-empty collections.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations show readOnlyHint=false and openWorldHint=true. The description adds significant behavioral context: that by default it sends UI defaults, and without them the server uses different preprocessing leading to worse models. It also explains that parameters are merged with UI defaults. This goes beyond the annotations, though it could mention job creation or cost implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (three sentences) and front-loaded with the main purpose. It packs critical information about defaults and warnings without excessive verbosity. However, the technical details could be slightly overwhelming for an agent, but overall it's efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the high parameter count (24) and complex default behavior, the description covers the key behavioral aspects: defaults, merging, and the noTrainingDefaults option. It does not explain how to use the result (e.g., job completion, model ID retrieval) but the tool name and context make the purpose clear. It is reasonably complete for an agent to use correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so each parameter already has a description. The tool description adds value by explaining the merging behavior for textFeatsCfg, createModelPars, and trainCtcPars, and the shortcut parameters (max_epochs, etc.). It clarifies the role of noTrainingDefaults, which is not detailed in the schema. This adds meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Start PyLaia HTR model training for a collection'. It specifies the resource (PyLaia model) and the action (training). The mention of UI defaults distinguishes it from siblings like transkribus_recog_train_htr_citlab, and the tool name itself is very specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use this tool (training PyLaia models) and the impact of the noTrainingDefaults flag. It warns that server defaults produce worse models, providing a clear context. However, it does not explicitly compare with other training tools (e.g., transkribus_recog_train_htr_citlab) or specify when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![transkribus-mcp-server MCP server](https://glama.ai/mcp/servers/lazyants/transkribus-mcp-server/badges/card.svg)](https://glama.ai/mcp/servers/lazyants/transkribus-mcp-server)

Score Badge

Copy to your README.md:

[![transkribus-mcp-server MCP server](https://glama.ai/mcp/servers/lazyants/transkribus-mcp-server/badges/score.svg)](https://glama.ai/mcp/servers/lazyants/transkribus-mcp-server)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/lazyants/transkribus-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server