lilbee
Server Quality Checklist
Latest release: v1.0.9
- Disambiguation3/5
Most tools are distinct, but several overlap semantically: status/wiki_status/crawl_status all report state, reset/wiki_wipe/settings_reset/clear_placement all delete or reset something, and catalog_browse/model_show both touch the model catalog. Descriptions help, but generic names like reset, add, sync, and status increase the chance an agent picks the wrong one.
Naming Consistency4/5The set mostly follows snake_case verb_noun naming, e.g. list_documents, set_placement, model_pull. However, verb placement is inconsistent (get_gpus vs settings_get, model_rm vs model_pull) and several single-word tools (reset, sync, add, init, crawl) break the pattern. Still, the overall style is predictable and readable.
Tool Count2/529 tools exceeds the 25-tool threshold and feels heavy even though the server covers multiple subdomains (indexing, crawling, wiki, models, settings, GPU placement). Several tools could be consolidated, such as the various status/get commands. The breadth is real, but the count is still too high for easy agent navigation.
Completeness4/5The tool surface covers the main lifecycles well: document add/sync/remove/list/export/import/search, crawl start/status/cancel, model list/show/pull/remove, settings CRUD, and placement get/preview/set/clear. The wiki layer is thin with only status and wipe, and there is no explicit wiki generation or document update tool, but these are minor gaps rather than blocking dead ends.
Average 3.8/5 across 29 of 29 tools scored. Lowest: 2.7/5.
See the Tool Scores section below for per-tool breakdowns.
- 2 of 2 community issues answered or closed in the last 6 months
- 392 commits in the last 12 weeks
- Last stable release on
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
This server has been verified by its author.
How to sync the server with GitHub?
Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.
To manually sync the server, click the "Sync Server" button in the MCP server admin interface.
How is the quality score calculated?
The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).
Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.
Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).
Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.
Tool Scores
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only says 'Show catalog and installed metadata' without disclosing side effects, permission needs, rate limits, or what exactly 'catalog and installed metadata' entails. The minimal description leaves the agent guessing about behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is concise and to the point. While it lacks detail, it is not overly verbose. Every word contributes to stating the tool's basic function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given low complexity (1 parameter, output schema exists), the description is minimally adequate. It explains the basic purpose but omits details about return values, parameter semantics, and how it differs from similar tools. The output schema reduces some burden, but more context would help.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single required parameter 'model' has no description in the input schema (0% coverage). The description does not clarify acceptable values, format (e.g., name, ID, path), or constraints. No additional meaning is provided beyond the schema's type string.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
Description uses verb 'Show' and resource 'catalog and installed metadata for a model ref,' clearly indicating the tool's function. It is specific enough to distinguish from siblings like 'model_list' (which lists models) and 'model_pull' (which downloads). However, it does not explicitly differentiate from similar inspection tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'model_list' or 'search.' No exclusions or prerequisites mentioned. The description simply states the action without contextual hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions 'writable settings' implying read-only settings cannot be reset, but it fails to clarify that only specified keys are reset (contradicted by the required 'keys' parameter which suggests per-key operation). No annotations exist, so the description must carry all behavioral info but is incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, which is efficient. However, it lacks additional structure (e.g., bullet points) that could improve readability, but given the simplicity, it is appropriately sized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the number of sibling tools and no annotations, the description should provide more context about scope (e.g., does it reset all writable settings or only specified ones?), but it does not. The existence of an output schema partially mitigates the need to describe return values, but overall the description is incomplete for a complex tool ecosystem.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description does not mention the 'keys' parameter at all, leaving its purpose and format entirely undocumented. The tool's behavior is implied as resetting settings, but how to specify which settings is missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Reset') and resource ('writable settings') with the outcome ('built-in defaults'). It clearly distinguishes from sibling tools like settings_list (read) and settings_get (read single), as reset is a mutation action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. For example, it does not explain when a reset is appropriate compared to updating individual settings via other tools, nor does it mention any prerequisites or side effects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only states the tool gets a value and metadata, but fails to mention side effects, auth requirements, rate limits, or behavior when the key does not exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at one sentence, front-loading the core purpose. However, it sacrifices valuable detail that could be included without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not detailed here), the description does not need to elaborate on return values. However, with only one parameter and no parameter description, along with missing usage guidelines and behavioral context, the description falls short of being fully complete for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage for the 'key' parameter, the description adds no additional meaning beyond the parameter name. It does not explain what constitutes a valid key, how to obtain it, or any format constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'a single setting', and the output 'current value + metadata'. This distinctly separates it from sibling tools like settings_list (list multiple) and settings_reset (reset).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as when to use settings_get over settings_list or other tools. There are no context signals or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It explains the meaning of enum-like values (e.g., max_fit: fits/tight/wont_run) but does not disclose pagination behavior, default sort, whether 'featured' and 'installed' are filters or flags, or what the response contains. The output schema exists but the description itself adds little behavioral context beyond value semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loads the main purpose in the first sentence, then uses a terse key: value format for parameter semantics. It is efficient, though the formatting is slightly telegraphic and could be clearer with a short sentence about how filters combine.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 9-parameter browse tool with no annotations and 0% schema coverage, the description covers the core filter dimensions but leaves limit, offset, search, featured, and installed unexplained. The output schema exists, so return values are partially covered, but an agent would still be guessing about pagination and filter semantics. This is a minimally viable description with clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does add meaning for four parameters (task, size, max_fit, sort) by listing allowed values, which is genuinely useful. However, five parameters (limit, offset, search, featured, installed) are left entirely undocumented, and the description does not explain how the parameters combine, so the compensation is only partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb and resource ('Browse the model catalog') and enumerates the key filter dimensions (task, size, max_fit, sort), which distinguishes it from sibling tools like model_list and model_show. However, it does not explicitly contrast itself with those siblings, so it loses a point for not naming the alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by listing the filter dimensions and their allowed values, giving an agent a sense of how to narrow a catalog browse. It does not state when to prefer this over model_list or model_show, nor does it mention any exclusions or prerequisites, so the guidance is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries burden. States 'No changes made' which signals non-destructive preview. Does not disclose permissions, rate limits, or error behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, direct sentences. No unnecessary words. Could be slightly more informative without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple preview tool with one optional parameter and output schema. However, lacks explanation of the output or any error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must add meaning. Only says 'placement spec (or auto, when omitted)', which clarifies that the object represents a placement spec but provides no structure or constraints for the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'preview' and resource 'placement spec'. Indicates non-mutating behavior ('No changes made'). Lacks explicit differentiation from siblings like 'clear_placement' but distinct enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies safe preview usage via 'No changes made', but no explicit guidance on when to use vs alternatives like 'add' or 'clear_placement'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It discloses the return fields but omits behavioral traits like idempotency, rate limits, or error handling. Adequate for a simple status poll, but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with inline code formatting. No wasted words; efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter and existence of an output schema, the description covers the basics. However, it could mention it is meant to be used after a crawl to poll progress, enhancing context completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It mentions polling 'by id' and the parameter is named 'task_id', making the mapping clear. No additional syntax or constraints provided, but sufficient for a single parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it polls a crawl task by id and returns status, pages, and error, distinguishing it from sibling tools like 'crawl' for initiating tasks. However, the verb 'poll' could be more explicit about checking status after initiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., 'crawl' for starting a task, 'search' for searching). It is implied for checking status after a crawl, but lacks explicit when-not or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden. It explains filter parameters but does not disclose results format, pagination, performance, or any side effects. Minimal behavioral context beyond basic functionality.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with clear front-loading. Every word adds value, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple and output schema exists, so description is mostly complete. Provides necessary filter context. Could benefit from brief note on result limitation or ordering, but adequate for a list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It defines 'source' as native/remote and 'task' as filter by role, adding meaning but remaining somewhat vague (e.g., 'role' not defined). Partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Explicitly states 'List installed models' with clear verb and resource. Distinguishes from sibling tools like 'search' or 'sync' by focusing on listing installed models with filtering options.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description implies listing usage but does not mention exclusions or compare with siblings like 'search' or 'crawl'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It only states the basic action without mentioning side effects, reversibility, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, minimal sentence that conveys the core purpose without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no parameters and an output schema, the description is mostly complete but lacks context on prerequisites or result details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so baseline score of 4 applies; no additional parameter description needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (clear) and the resource (manual placement), and distinguishes it from siblings like preview_placement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like add or remove, nor any prerequisites or contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses key behaviors: replaces existing copies and detaches imported sources from sync. However, it lacks details on authorization, rate limits, or potential side effects on other data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no fluff. The first sentence states the primary purpose, and the second adds essential behavioral context. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and the tool's moderate complexity, the description covers the main action, replacement, and detachment. It lacks details on re-embedding process and prerequisites but is generally sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It partially explains the 'dataset' parameter (the dataset to import) but does not mention the 'fmt' parameter at all, leaving its purpose unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool imports a per-page text dataset and re-embeds it under the current model. It distinguishes the tool from siblings like export_dataset by specifying the action and additional behavior of replacing copies and detaching sources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., crawl, add). The description does not specify prerequisites or preferred contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It mentions streaming progress and parameter effects but omits important details like auth requirements, side effects, or error handling. For a download tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two lines with no redundant information. The first sentence states the core purpose, followed by precise parameter explanations. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a download tool with streaming and no annotations, the description covers basic usage and parameter meanings. However, it lacks details on return format (though output schema exists), error scenarios, or concurrency/rate limits. Adequate but with clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates by explaining source as 'native (GGUF) or remote (SDK)' and allow_unsupported as 'overrides the supported-architecture refusal.' Model parameter is not elaborated, but overall it adds significant meaning beyond the schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Download a model and stream progress,' which is a specific verb-resource pair. It distinguishes from siblings like model_list and model_show by focusing on downloading rather than listing or inspecting.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool over alternatives such as model_show or search. The description only explains parameter details but lacks context like prerequisites or use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the destructive nature (remove) and the limitation to certain sources, but lacks details on whether removal is irreversible, permissions needed, or any confirmation steps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Description is very concise at two sentences, front-loading the purpose. It earns its keep by covering scope and limitations, but could be slightly more structured for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool (removal with two params) and presence of output schema, the description covers the core action and constraints. However, it lacks parameter descriptions, which is a notable gap for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage and the description adds no explanation for the 'model' and 'source' parameters. The description does not clarify what values are expected or how 'source' affects behavior, leaving a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Remove an installed model' with a specific verb and resource. It distinguishes from siblings by specifying it only works on native GGUF models from 'lilbee' (likely a typo for 'libbee' or similar), and notes that Ollama/LM Studio models are read-only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context for when to use (removing models) and an exclusion (Ollama/LM Studio models are read-only). However, it does not mention alternatives for removing other model types or any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes core behavior (write to file, no vectors) but does not disclose side effects (e.g., read-only, no modification of original data). No annotations provided, so description carries burden but only partially addresses it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-loaded with main purpose and parameters. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main purpose and two parameter details, but lacks usage context and behavioral safety. Output schema exists (not shown) so return values not needed. With no annotations, description is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Explains fmt (parquet/jsonl, infers from suffix) and source (limits to one file) but does not describe output parameter beyond being the file path. Schema coverage is 0%, so description adds some value but leaves gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool writes a per-page dataset to a file, specifies no vectors, and implies export action. Distinguishes from import_dataset (sibling) by nature of operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for exporting dataset to file in parquet/jsonl optionally limited to one source. No explicit when-to-use or alternative guidance, but context suggests it's for file output.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses atomicity, rollback on validation error, persistence to config.toml, and the return format. It does not mention authentication requirements or what happens on success beyond the return, but overall provides good behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with key information (atomic update, rollback, persistence, return value). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (implied by has_output_schema: true), the description omits critical details about the 'updates' parameter structure, which is essential for correct invocation. The tool's complexity (atomicity, validation) demands more parameter documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has a single required parameter 'updates' of type object, but the description adds no detail about its expected structure (keys, value types, required fields). Schema description coverage is 0%, so the agent has no guidance on how to construct the updates object.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('atomically update writable settings'), the persistence target ('config.toml'), and the return value. It distinguishes from sibling tools like settings_get, settings_list, and settings_reset by focusing on atomic updates with rollback.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied ('update writable settings') but no explicit guidance on when to use this tool over alternatives like settings_reset or when not to use it. No prerequisites or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. 'Show' implies read-only, but doesn't explicitly state non-destructiveness, permissions needed, or side effects. Adequate for a simple status tool but lacks explicit behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence of 6 words, front-loading the action and scope. No wasted words; every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and presence of an output schema, the description sufficiently conveys what the tool does. Could mention it is read-only, but the word 'Show' implies that. Competently covers the essentials.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, so schema coverage is 100%. Baseline for 0 parameters is 4. Description adds no param info, which is appropriate given no parameters exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description states a specific verb ('Show') and clearly lists the resources ('indexed documents, configuration, and chunk counts'). It distinguishes itself from sibling tools like 'search' or 'sync' by indicating it provides a summary view.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., 'search' for querying, 'list_documents' for document listing). No prerequisites, exclusions, or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing side effects. It does so explicitly for each flag: force_rebuild 'drops every table', retry_skipped 'clears failed-file skip markers', and prune_ignored 'drops sources' without dropping the store. It also notes the default behavior of prune_ignored. This provides meaningful behavioral safety information that is absent from the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused paragraph that front-loads the primary action and then explains each flag using backticks for clarity. It is concise without being overly terse, and every sentence contributes to describing behavior. The only minor inefficiency is the lack of a visual break between purpose and parameter details, but it remains well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description appropriately focuses on the tool's purpose and parameter effects. It covers all three parameters and their edge cases but does not mention default behavior when no flags are passed, potential side effects on existing data beyond force_rebuild, or concurrency/asynchronous behavior. These are likely captured in the output schema, so the description is largely complete for the agent's needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description is the sole source of parameter meaning. It explains all three boolean parameters in detail with concrete effects and consequences. This fully compensates for the schema's lack of descriptions, giving the agent clear guidance on what each flag does.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Sync the documents directory into the vector store.' This clearly states the tool's core function. While it doesn't explicitly name sibling alternatives, the phrasing is distinct enough (e.g., vs. 'add' or 'crawl') to avoid confusion, though an explicit differentiation would warrant a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains what the three boolean flags do but does not provide any context on when to use this tool versus siblings like 'init', 'add', or 'crawl'. There is no mention of typical use cases, prerequisites, or conditions that would make 'sync' the appropriate choice over others. This leaves the agent to infer usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral disclosure burden. It does disclose the non-blocking nature, the depth semantics (0, N, null), and render_mode options. However, it omits behavior for max_pages and include_subdomains, and does not mention side effects, resource implications, or how the task_id is returned beyond implying it exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three compact lines with no filler. The main action and polling workflow are front-loaded, followed by parameter semantics. Every sentence adds information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core workflow and two of five parameters, and an output schema exists so return values need not be described. But for a no-annotation tool with 5 parameters, omitting max_pages and include_subdomains leaves the agent without enough context to use the full tool correctly, and the null-depth note is inconsistent with the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds meaningful semantics for depth (0 default, N levels, null whole site) and restates render_mode options. But url, max_pages, and include_subdomains are left unexplained, and the 'null' depth value conflicts with the schema's integer type, creating ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific action and resource: 'Start a non-blocking crawl.' It also distinguishes itself from the sibling crawl_status by directing the agent to poll via crawl_status(task_id), and from crawl_cancel implicitly by describing the start workflow. The depth and render_mode semantics further clarify what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives a clear usage context: use this tool to initiate an asynchronous crawl, then poll with crawl_status. It does not explicitly state when to prefer crawl over search or sync, but it names the companion polling tool and implies the start-then-poll workflow, which is clear context without exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It reveals that pages already saved are kept, which is useful. But it does not mention other behaviors like whether the cancel is immediate, error handling, or permission requirements, leaving some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, front-loaded sentences with zero waste. Every word earns its place, clearly stating the action and a key behavioral consequence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple cancel operation, the description covers the main purpose and the key side effect (saved pages kept). An output schema exists, so return values need not be explained. Minor missing details about edge cases (e.g., task not found or already finished) are acceptable given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one required parameter, task_id, with no description. The tool description does not explain what task_id is or where to obtain it (e.g., from a crawl call). With 0% schema description coverage, the description should compensate, but it doesn't.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action ('Stop a running crawl') on a specific resource ('crawl started by ``crawl``'), which distinguishes it from sibling tools like crawl and crawl_status. The verb and resource are unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It implies when to use (for crawls started by `crawl` that are still running) and provides a clear context. However, it does not explicitly mention alternatives or when not to use, such as checking status first with crawl_status.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the initialization and session switching behavior, but does not address idempotency, error cases, or effects if the knowledge base already exists. No annotations are provided to supplement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two clear sentences, no redundancy, and the key behavior is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic purpose but lacks details on prerequisites, error cases, and integration with other tools. The presence of an output schema may compensate for return values, but overall completeness is limited.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds the default behavior for an empty path beyond the schema's simple string type, but does not detail valid paths, error conditions, or other constraints. Given 0% schema coverage, more parameter detail would improve.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool initializes a local .lilbee/ knowledge base and switches the session, with default to cwd. This specific verb+resource distinguishes it from sibling tools like search or sync.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used before other calls to set the session knowledge base, but lacks explicit when-not-to-use guidance or comparisons with alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Indicates a read-only list operation, but does not disclose potential issues like large result sets, pagination, or any side effects; lacks depth given no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no waste; direct and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, parameterless tool with an output schema, the description is sufficient to understand its purpose and output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so the description adequately covers the schema; no additional value needed beyond the trivial schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all indexed documents and includes chunk counts, distinguishing it from siblings like search or add.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., search, status) or any prerequisites; lacks explicit context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It does disclose a key behavioral trait: URLs are fetched as single pages, not crawled, and paths are absolute and resolve on the server's machine. It also mentions the sync requirement. However, it doesn't disclose side effects (e.g., whether existing files are overwritten) or what happens with OCR or render mode settings, and it doesn't mention error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: the first states purpose and sync requirement, the second explains path resolution, and the third clarifies URL behavior vs crawl. Every sentence adds critical information with zero filler. It is front-loaded with the primary action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters and an output schema exists, the description covers the most important operational context (path resolution, URL behavior, sync requirement). The gaps are explanation of parameters like 'force' and OCR options, but those are less critical to basic invocation. The presence of an output schema likely explains return values, so the description doesn't need to. A 4 is justified because it handles the trickiest contexts well, though it could mention what the return value indicates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. The description partially does by implying that 'paths' is the main argument, and it clarifies how URLs are handled (single pages). However, it doesn't explain the semantics of 'force', 'enable_ocr', 'ocr_timeout', or 'render_mode' beyond what the schema minimally hints (e.g., 'render_mode' enum text). A 3 seems right because it adds some meaning for 'paths' and 'render_mode' indirectly but leaves key parameters unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('add') and resource ('files, directories, or URLs to the knowledge base'), and clarifies the follow-up action ('then sync'). It also distinguishes itself from the sibling tool 'crawl' by noting that URLs are fetched as single pages, which clearly separates its role from crawl's site-wide crawling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use this tool (when adding files, directories, or single URLs) and notes an important environment constraint (paths resolve on the lilbee machine, not the caller's when remote). It also provides an explicit exclusion: use 'crawl' for sites. However, it does not detail when to prefer alternative tools like 'import_dataset' or 'sync'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries full burden. It discloses destructive behavior (delete all documents and indexed data) and the confirm requirement. But lacks details on reversibility or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no fluff. Every part provides essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple destructive tool with one parameter, the description covers the core function and key requirement. Does not mention output, but completeness is reasonable given output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (confirm) with 0% schema coverage. Description compensates by stating it requires confirm=true, adding meaning beyond the boolean type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Factory reset: delete all documents and indexed data', using a specific verb and resource. It implicitly distinguishes from sibling tools like 'settings_reset' by indicating a full reset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
States the requirement 'Requires confirm=true', which is a usage constraint. However, it does not provide guidance on when to use this tool versus alternatives like 'settings_reset' or 'remove'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the burden. It clearly indicates a read-only operation listing GPUs with memory info. No destructive or side effects are suggested, and the behavior is transparent for such a simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys all necessary information without extraneous words. It is perfectly sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and an output schema present, the description fully covers the tool's purpose and behavior. It explains what is listed and provides an equivalent API reference, leaving no gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so schema coverage is 100%. The description adds meaningful context about the return values (free/total VRAM) beyond the schema, justifying a baseline of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists detected GPUs with free/total VRAM, and provides the equivalent HTTP endpoint. This distinguishes it from sibling tools which are unrelated to GPU listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this tool is for checking GPU availability, but does not explicitly state when to use it versus alternatives or any prerequisites. However, due to its simplicity (no parameters), usage is largely self-evident.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It correctly implies a read-only operation without side effects. It does not specify edge cases but is adequate for a simple getter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, succinct sentence that front-loads the key action and resource. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with no parameters and an output schema, the description provides all necessary context: it returns the current effective placement. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so schema coverage is 100%. The description adds no parameter-specific value, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Show the current effective multi-GPU model placement.' It uses a specific verb ('Show') and resource ('multi-GPU model placement'), and distinguishes itself from siblings like set_placement and clear_placement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
While no explicit when/when-not guidance is given, the tool's purpose is straightforward and the name implies its usage relative to siblings (set, clear, preview). Context is clear enough for an agent to infer when to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It explicitly states that source files are never deleted and explains matching behavior for folders and globs. This covers key behavioral traits, though rate limits or reversibility are not addressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, no wasted words. The first sentence clearly states purpose, and the second adds critical behavioral context. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter and the existence of an output schema, the description covers input semantics and key behavioral notes. It lacks any mention of return values, but the output schema likely provides that. Minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% for the 'names' parameter, but the description thoroughly explains that names can be source names, folder names, or glob patterns, with specific semantics for each. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The verb 'Remove' and resource 'documents from the index' are clear. It specifies the input types (source name, folder, glob pattern) and distinguishes from sibling tools like 'add' and 'sync'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (to remove indexed documents) but does not explicitly state when not to use it or mention alternative tools. No clear guidance on prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes return format (chunks with citations) and scope semantics. No annotations provided, so description carries full burden; lacks details on top_k range or rate limits, but adds useful context beyond schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences plus a scope note; no fluff. Front-loaded purpose and guidelines.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, scope, and output format. Lacks top_k default/range and any mention of permissions, but given output schema exists, it's sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% coverage, so description compensates by explaining scope values and default. Query is assumed, top_k is not explained but is self-explanatory. Still, partial coverage justifies 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches indexed documents, code, and crawled pages, distinguishing it from web-fetch or file-read tools. It specifies the verb and resources, with a clear preference note.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends this tool over web-fetch or file-read, and clarifies scope usage with default values and wiki limitation. Missing when-not-to-use conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the output fields (value, default, type, help, choices) and parameter behavior. Despite no annotations, it gives sufficient transparency for safe usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two succinct lines, front-loaded with the main purpose, and each word adds value. No redundancy or waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (single optional parameter, list operation), the description covers output structure and parameter behavior. No missing critical details like authentication or pagination, which are likely not needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'group' has zero coverage in schema but the description adds full semantics: filtering behavior, case-insensitivity, and default when empty. This compensates well for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and the resource ('writable lilbee settings'), with specific details about output fields. The name and description distinguish it from siblings like settings_get and settings_reset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the optional group parameter's behavior (case-insensitive filter, empty returns all), providing clear context for when to use it. While it doesn't explicitly contrast with sibling tools, the purpose is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral disclosure burden. It discloses an important trait: it is registered even when the wiki is disabled, similar to an HTTP status route. This gives context about availability and what the tool can reveal, which is sufficient for a status check tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the main purpose front-loaded clearly in the first sentence. The second sentence adds a meaningful caveat about disabled state without unnecessary elaboration. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description need not explain return values. It covers the essential purpose and the notable behavior about disabled state. It could mention what the response looks like, but for a simple status tool this is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema coverage is trivially 100%. The description correctly omits parameter details. Baseline for a zero-parameter tool is 4, and no additional explanation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Show wiki layer status' with specific contents (page counts, recent lint issues). It distinguishes itself from sibling tools like 'status' by focusing on the wiki layer, and the name reinforces this specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explains a key usage context: even when the wiki is disabled, this tool remains registered so callers can read the disabled state. This implies using this tool when you need wiki status regardless of enabled state, but it does not explicitly compare to alternatives like 'status'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool deletes all generated wiki pages and their indexed rows, and that it requires confirmation (confirm=true). It also explains its availability even when the wiki is disabled, which is valuable behavioral context. With no annotations, this description carries the full burden and does a good job conveying the destructive and conditional nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the action and requirement, the second provides rationale for the tool's availability. Every sentence adds value and it is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple destructive tool with one parameter, the description covers the action, the confirmation requirement, and a specific use case (even when wiki disabled). It doesn't explain return values or permissions, but an output schema is present, so that's not required. The description is complete enough for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema only defines a boolean 'confirm' with no description, but the description explicitly instructs to 'Pass confirm=true,' clarifying the parameter's role and required value. This adds meaning beyond the schema, though it could further explain the behavior when confirm is false.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete every generated wiki page and its indexed rows.' This is a specific verb (delete), resource (wiki pages and indexed rows), and scope (every). It distinguishes the tool from siblings like 'remove' or 'reset' by focusing on generated wiki pages.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that the tool is 'Registered even with the wiki disabled, because turning the setting off leaves the pages generated earlier in place,' providing clear context for when to use it. It also instructs to 'Pass confirm=true,' indicating the required action. However, it doesn't explicitly mention when not to use it or name alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses that the operation persists to config (indicating state change) and details the spec structure. It does not mention permissions, side effects on other placements, or error handling, but the core behavioral traits are well covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded, stating the purpose in the first sentence. It includes a compact example and additional details in a few sentences without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists, the description does not need to explain return values. It covers the spec format, optional fields, and the persistence behavior. Minor gaps include lack of validation rules or error scenarios, but overall it provides sufficient context for a single-parameter set operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides only an object type with no inner structure, resulting in 0% coverage. The description fully compensates by specifying the expected format: a mapping from role to an object containing devices array and optional tensor_split array, with a concrete JSON example. This adds essential meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Set and apply'), the resource ('manual multi-GPU placement spec'), and the key behavioral trait ('persists to config'). It effectively distinguishes this tool from its siblings like get_placement, preview_placement, and clear_placement by focusing on setting/applying rather than retrieving or clearing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear example of the expected format and explains how to specify roles and devices, including the optional tensor_split. It mentions that omitting a role leaves it auto-placed. However, it does not explicitly state when this tool should be avoided (e.g., use preview_placement for dry-run) or list alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
GitHub Badge
Glama performs regular codebase and documentation scans to:
- Confirm that the MCP server is working as expected.
- Confirm that there are no obvious security issues.
- Evaluate tool definition quality.
Our badge communicates server capabilities, safety, and installation instructions.
Card Badge
Copy to your README.md:
Score Badge
Copy to your README.md: