lilbee
Server Quality Checklist
Latest release: v1.0.4
- Disambiguation4/5
Tools are grouped into distinct domains (knowledge base, settings, models, GPU placement) and each tool has a clear purpose. However, some overlap exists between 'sync' and 'crawl' as both ingest content, though descriptions differentiate them.
Naming Consistency3/5Mixed naming conventions: verb_noun (list_documents, search, add) and noun_verb (settings_list, catalog_browse) are used interchangeably. This inconsistency may confuse agents looking for predictable patterns.
Tool Count2/526 tools is excessive for a single server, especially with 5 tools dedicated to GPU placement and 4 to settings. While the server covers multiple domains, the sheer number can overwhelm agents and suggests over-scoping.
Completeness4/5The tool set provides comprehensive coverage for knowledge base management (CRUD-like operations), model management, settings, and GPU placement. Minor gaps exist, such as no explicit update tool for documents, but import_dataset can replace them.
Average 3.7/5 across 26 of 26 tools scored. Lowest: 2.5/5.
See the Tool Scores section below for per-tool breakdowns.
- 2 of 2 community issues answered or closed in the last 6 months
- 1100 commits in the last 12 weeks
- Last stable release on
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
This repository includes a glama.json configuration file.
This server has been verified by its author.
How to sync the server with GitHub?
Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.
To manually sync the server, click the "Sync Server" button in the MCP server admin interface.
How is the quality score calculated?
The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).
Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.
Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).
Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.
Tool Scores
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It only says 'browse' and lists parameters, but does not indicate whether the operation is read-only, what the return format is, or whether pagination is handled. No safety or side-effect information is provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief and front-loaded with the main purpose. It efficiently lists parameter options without unnecessary words. Minor formatting issues (double backticks) do not detract significantly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 8 parameters and an output schema, but the description only covers 3 parameters and offers no usage context. It fails to explain filtering, pagination, or how it relates to sibling tools, leaving the agent under-informed for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 8 parameters with 0% description coverage. The description adds meaning for task, size, and sort by listing allowed values, but leaves limit, offset, search, featured, and installed completely undocumented. This partial compensation is insufficient for a low-coverage schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool browses the lilbee model catalog, with a specific verb and resource. However, it does not distinguish this from sibling tools such as model_list or search, so it lacks differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like model_list or search. The description only explains the tool's basic function without stating context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only says 'Show catalog and installed metadata' without disclosing side effects, permission needs, rate limits, or what exactly 'catalog and installed metadata' entails. The minimal description leaves the agent guessing about behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is concise and to the point. While it lacks detail, it is not overly verbose. Every word contributes to stating the tool's basic function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given low complexity (1 parameter, output schema exists), the description is minimally adequate. It explains the basic purpose but omits details about return values, parameter semantics, and how it differs from similar tools. The output schema reduces some burden, but more context would help.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single required parameter 'model' has no description in the input schema (0% coverage). The description does not clarify acceptable values, format (e.g., name, ID, path), or constraints. No additional meaning is provided beyond the schema's type string.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
Description uses verb 'Show' and resource 'catalog and installed metadata for a model ref,' clearly indicating the tool's function. It is specific enough to distinguish from siblings like 'model_list' (which lists models) and 'model_pull' (which downloads). However, it does not explicitly differentiate from similar inspection tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'model_list' or 'search.' No exclusions or prerequisites mentioned. The description simply states the action without contextual hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description mentions 'writable settings' implying read-only settings cannot be reset, but it fails to clarify that only specified keys are reset (contradicted by the required 'keys' parameter which suggests per-key operation). No annotations exist, so the description must carry all behavioral info but is incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, which is efficient. However, it lacks additional structure (e.g., bullet points) that could improve readability, but given the simplicity, it is appropriately sized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the number of sibling tools and no annotations, the description should provide more context about scope (e.g., does it reset all writable settings or only specified ones?), but it does not. The existence of an output schema partially mitigates the need to describe return values, but overall the description is incomplete for a complex tool ecosystem.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description does not mention the 'keys' parameter at all, leaving its purpose and format entirely undocumented. The tool's behavior is implied as resetting settings, but how to specify which settings is missing.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Reset') and resource ('writable settings') with the outcome ('built-in defaults'). It clearly distinguishes from sibling tools like settings_list (read) and settings_get (read single), as reset is a mutation action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. For example, it does not explain when a reset is appropriate compared to updating individual settings via other tools, nor does it mention any prerequisites or side effects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for behavioral disclosure. It only states the tool gets a value and metadata, but fails to mention side effects, auth requirements, rate limits, or behavior when the key does not exist.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at one sentence, front-loading the core purpose. However, it sacrifices valuable detail that could be included without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema (not detailed here), the description does not need to elaborate on return values. However, with only one parameter and no parameter description, along with missing usage guidelines and behavioral context, the description falls short of being fully complete for an agent to use effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage for the 'key' parameter, the description adds no additional meaning beyond the parameter name. It does not explain what constitutes a valid key, how to obtain it, or any format constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'a single setting', and the output 'current value + metadata'. This distinctly separates it from sibling tools like settings_list (list multiple) and settings_reset (reset).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as when to use settings_get over settings_list or other tools. There are no context signals or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries burden. States 'No changes made' which signals non-destructive preview. Does not disclose permissions, rate limits, or error behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, direct sentences. No unnecessary words. Could be slightly more informative without harming conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple preview tool with one optional parameter and output schema. However, lacks explanation of the output or any error conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must add meaning. Only says 'placement spec (or auto, when omitted)', which clarifies that the object represents a placement spec but provides no structure or constraints for the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'preview' and resource 'placement spec'. Indicates non-mutating behavior ('No changes made'). Lacks explicit differentiation from siblings like 'clear_placement' but distinct enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies safe preview usage via 'No changes made', but no explicit guidance on when to use vs alternatives like 'add' or 'clear_placement'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that URLs are crawled as markdown and that paths must be absolute, and implies a sync occurs. However, it does not detail side effects, permissions, or what happens with duplicates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences. The first sentence front-loads the purpose, and the second adds critical constraints. No superfluous text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 5 parameters undocumented and an output schema present but not described, the description is incomplete. It fails to explain the sync behavior, parameter roles, or return values. The tool's complexity demands more detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description does not explain the parameters (force, enable_ocr, ocr_timeout, render_mode). Only 'paths' and URLs are mentioned. The description adds little meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description states exactly what the tool does: 'Add files, directories, or URLs to the knowledge base, then sync.' It uses a specific verb ('add') and resource ('knowledge base'), and differentiates from siblings like 'sync' and 'crawl' by focusing on addition.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. It only gives a constraint ('Paths must be absolute') but does not explain when to choose 'add' over sibling tools like 'crawl' or 'sync'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden. It discloses the return fields but omits behavioral traits like idempotency, rate limits, or error handling. Adequate for a simple status poll, but minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with inline code formatting. No wasted words; efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter and existence of an output schema, the description covers the basics. However, it could mention it is meant to be used after a crawl to poll progress, enhancing context completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It mentions polling 'by id' and the parameter is named 'task_id', making the mapping clear. No additional syntax or constraints provided, but sufficient for a single parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it polls a crawl task by id and returns status, pages, and error, distinguishing it from sibling tools like 'crawl' for initiating tasks. However, the verb 'poll' could be more explicit about checking status after initiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., 'crawl' for starting a task, 'search' for searching). It is implied for checking status after a crawl, but lacks explicit when-not or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden. It explains filter parameters but does not disclose results format, pagination, performance, or any side effects. Minimal behavioral context beyond basic functionality.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with clear front-loading. Every word adds value, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool is simple and output schema exists, so description is mostly complete. Provides necessary filter context. Could benefit from brief note on result limitation or ordering, but adequate for a list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It defines 'source' as native/remote and 'task' as filter by role, adding meaning but remaining somewhat vague (e.g., 'role' not defined). Partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Explicitly states 'List installed models' with clear verb and resource. Distinguishes from sibling tools like 'search' or 'sync' by focusing on listing installed models with filtering options.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. The description implies listing usage but does not mention exclusions or compare with siblings like 'search' or 'crawl'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the effects of force_rebuild (drops tables and re-ingests) and retry_skipped (clears skip markers). However, it does not describe other important behaviors such as whether the sync is incremental, what happens to files removed from the directory, or whether the operation is idempotent. The description adds some transparency but leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two short sentences. It front-loads the main purpose and immediately provides parameter explanations. There is no redundant or extraneous information; every word earns its place. This is a model of conciseness for a simple tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (two boolean parameters, no annotations, output schema present but unknown), the description covers the parameters well but lacks details on overall sync behavior (e.g., incremental vs. full sync, handling of deleted files, potential data loss). The output schema exists but the description does not need to explain return values. Still, the description leaves room for ambiguity about the core operation, so completeness is adequate but not excellent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains both parameters: force_rebuild (drops every table and re-ingests) and retry_skipped (clears failed-file skip markers). This adds meaning beyond the schema, which only provides type and name. The explanations are concise and actionable, earning a high score despite the lack of comprehensive parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (sync) and the resource (documents directory into vector store). It distinguishes from siblings like 'add' and 'crawl' by focusing on synchronization rather than individual additions or crawling. However, it does not specify the sync direction (e.g., one-way or bidirectional), which leaves some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives such as 'add' or 'crawl'. It explains the two boolean parameters but lacks explicit context on prerequisites, typical use cases, or situations to avoid using this tool. This leaves the agent without decision support for tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It only states the basic action without mentioning side effects, reversibility, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, minimal sentence that conveys the core purpose without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no parameters and an output schema, the description is mostly complete but lacks context on prerequisites or result details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so baseline score of 4 applies; no additional parameter description needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (clear) and the resource (manual placement), and distinguishes it from siblings like preview_placement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like add or remove, nor any prerequisites or contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses key behaviors: replaces existing copies and detaches imported sources from sync. However, it lacks details on authorization, rate limits, or potential side effects on other data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no fluff. The first sentence states the primary purpose, and the second adds essential behavioral context. Front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and the tool's moderate complexity, the description covers the main action, replacement, and detachment. It lacks details on re-embedding process and prerequisites but is generally sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It partially explains the 'dataset' parameter (the dataset to import) but does not mention the 'fmt' parameter at all, leaving its purpose unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool imports a per-page text dataset and re-embeds it under the current model. It distinguishes the tool from siblings like export_dataset by specifying the action and additional behavior of replacing copies and detaching sources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives (e.g., crawl, add). The description does not specify prerequisites or preferred contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must fully disclose behavior. It mentions streaming progress and parameter effects but omits important details like auth requirements, side effects, or error handling. For a download tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: two lines with no redundant information. The first sentence states the core purpose, followed by precise parameter explanations. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a download tool with streaming and no annotations, the description covers basic usage and parameter meanings. However, it lacks details on return format (though output schema exists), error scenarios, or concurrency/rate limits. Adequate but with clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates by explaining source as 'native (GGUF) or remote (SDK)' and allow_unsupported as 'overrides the supported-architecture refusal.' Model parameter is not elaborated, but overall it adds significant meaning beyond the schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Download a model and stream progress,' which is a specific verb-resource pair. It distinguishes from siblings like model_list and model_show by focusing on downloading rather than listing or inspecting.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool over alternatives such as model_show or search. The description only explains parameter details but lacks context like prerequisites or use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the destructive nature (remove) and the limitation to certain sources, but lacks details on whether removal is irreversible, permissions needed, or any confirmation steps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
Description is very concise at two sentences, front-loading the purpose. It earns its keep by covering scope and limitations, but could be slightly more structured for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple nature of the tool (removal with two params) and presence of output schema, the description covers the core action and constraints. However, it lacks parameter descriptions, which is a notable gap for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% description coverage and the description adds no explanation for the 'model' and 'source' parameters. The description does not clarify what values are expected or how 'source' affects behavior, leaving a significant gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Remove an installed model' with a specific verb and resource. It distinguishes from siblings by specifying it only works on native GGUF models from 'lilbee' (likely a typo for 'libbee' or similar), and notes that Ollama/LM Studio models are read-only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context for when to use (removing models) and an exclusion (Ollama/LM Studio models are read-only). However, it does not mention alternatives for removing other model types or any prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes core behavior (write to file, no vectors) but does not disclose side effects (e.g., read-only, no modification of original data). No annotations provided, so description carries burden but only partially addresses it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-loaded with main purpose and parameters. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main purpose and two parameter details, but lacks usage context and behavioral safety. Output schema exists (not shown) so return values not needed. With no annotations, description is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Explains fmt (parquet/jsonl, infers from suffix) and source (limits to one file) but does not describe output parameter beyond being the file path. Schema coverage is 0%, so description adds some value but leaves gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool writes a per-page dataset to a file, specifies no vectors, and implies export action. Distinguishes from import_dataset (sibling) by nature of operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies use for exporting dataset to file in parquet/jsonl optionally limited to one source. No explicit when-to-use or alternative guidance, but context suggests it's for file output.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It mentions 'non-blocking' and depth behavior, which adds value. However, it does not cover other important aspects like idempotency, permissions, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two sentences, no fluff. The key points (non-blocking, depth, render_mode) are front-loaded. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers asynchronous behavior, depth, and render mode, but omits 'url', 'max_pages', and 'include_subdomains'. Despite having an output schema, the missing parameter descriptions reduce completeness. Given 5 params and no annotations, more detail is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%. The description adds meaning for 'depth' (whole site vs single URL) and 'render_mode' (http vs browser), but other parameters like 'url', 'max_pages', and 'include_subdomains' remain unexplained. This is insufficient to compensate for the lack of schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'start' and the resource 'crawl', and explains it is non-blocking and how to poll for results via 'crawl_status'. It distinguishes from the sibling 'crawl_status' tool. Depth and render_mode are also clarified.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides usage context: non-blocking, poll via crawl_status, and depth values (None vs 0). It implies when to use different depths but lacks explicit when-not-to-use or alternatives beyond the sibling.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses atomicity, rollback on validation error, persistence to config.toml, and the return format. It does not mention authentication requirements or what happens on success beyond the return, but overall provides good behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with key information (atomic update, rollback, persistence, return value). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema (implied by has_output_schema: true), the description omits critical details about the 'updates' parameter structure, which is essential for correct invocation. The tool's complexity (atomicity, validation) demands more parameter documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has a single required parameter 'updates' of type object, but the description adds no detail about its expected structure (keys, value types, required fields). Schema description coverage is 0%, so the agent has no guidance on how to construct the updates object.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('atomically update writable settings'), the persistence target ('config.toml'), and the return value. It distinguishes from sibling tools like settings_get, settings_list, and settings_reset by focusing on atomic updates with rollback.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied ('update writable settings') but no explicit guidance on when to use this tool over alternatives like settings_reset or when not to use it. No prerequisites or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. 'Show' implies read-only, but doesn't explicitly state non-destructiveness, permissions needed, or side effects. Adequate for a simple status tool but lacks explicit behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence of 6 words, front-loading the action and scope. No wasted words; every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and presence of an output schema, the description sufficiently conveys what the tool does. Could mention it is read-only, but the word 'Show' implies that. Competently covers the essentials.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, so schema coverage is 100%. Baseline for 0 parameters is 4. Description adds no param info, which is appropriate given no parameters exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description states a specific verb ('Show') and clearly lists the resources ('indexed documents, configuration, and chunk counts'). It distinguishes itself from sibling tools like 'search' or 'sync' by indicating it provides a summary view.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., 'search' for querying, 'list_documents' for document listing). No prerequisites, exclusions, or context provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It reveals that pages already saved are kept, which is useful. But it does not mention other behaviors like whether the cancel is immediate, error handling, or permission requirements, leaving some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two short, front-loaded sentences with zero waste. Every word earns its place, clearly stating the action and a key behavioral consequence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple cancel operation, the description covers the main purpose and the key side effect (saved pages kept). An output schema exists, so return values need not be explained. Minor missing details about edge cases (e.g., task not found or already finished) are acceptable given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one required parameter, task_id, with no description. The tool description does not explain what task_id is or where to obtain it (e.g., from a crawl call). With 0% schema description coverage, the description should compensate, but it doesn't.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action ('Stop a running crawl') on a specific resource ('crawl started by ``crawl``'), which distinguishes it from sibling tools like crawl and crawl_status. The verb and resource are unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It implies when to use (for crawls started by `crawl` that are still running) and provides a clear context. However, it does not explicitly mention alternatives or when not to use, such as checking status first with crawl_status.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the initialization and session switching behavior, but does not address idempotency, error cases, or effects if the knowledge base already exists. No annotations are provided to supplement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two clear sentences, no redundancy, and the key behavior is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the basic purpose but lacks details on prerequisites, error cases, and integration with other tools. The presence of an output schema may compensate for return values, but overall completeness is limited.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds the default behavior for an empty path beyond the schema's simple string type, but does not detail valid paths, error conditions, or other constraints. Given 0% schema coverage, more parameter detail would improve.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool initializes a local .lilbee/ knowledge base and switches the session, with default to cwd. This specific verb+resource distinguishes it from sibling tools like search or sync.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool should be used before other calls to set the session knowledge base, but lacks explicit when-not-to-use guidance or comparisons with alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Indicates a read-only list operation, but does not disclose potential issues like large result sets, pagination, or any side effects; lacks depth given no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no waste; direct and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, parameterless tool with an output schema, the description is sufficient to understand its purpose and output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist, so the description adequately covers the schema; no additional value needed beyond the trivial schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists all indexed documents and includes chunk counts, distinguishing it from siblings like search or add.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., search, status) or any prerequisites; lacks explicit context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries full burden. It discloses destructive behavior (delete all documents and indexed data) and the confirm requirement. But lacks details on reversibility or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no fluff. Every part provides essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple destructive tool with one parameter, the description covers the core function and key requirement. Does not mention output, but completeness is reasonable given output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (confirm) with 0% schema coverage. Description compensates by stating it requires confirm=true, adding meaning beyond the boolean type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Factory reset: delete all documents and indexed data', using a specific verb and resource. It implicitly distinguishes from sibling tools like 'settings_reset' by indicating a full reset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
States the requirement 'Requires confirm=true', which is a usage constraint. However, it does not provide guidance on when to use this tool versus alternatives like 'settings_reset' or 'remove'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries the burden. It clearly indicates a read-only operation listing GPUs with memory info. No destructive or side effects are suggested, and the behavior is transparent for such a simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that conveys all necessary information without extraneous words. It is perfectly sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no parameters and an output schema present, the description fully covers the tool's purpose and behavior. It explains what is listed and provides an equivalent API reference, leaving no gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, so schema coverage is 100%. The description adds meaningful context about the return values (free/total VRAM) beyond the schema, justifying a baseline of 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists detected GPUs with free/total VRAM, and provides the equivalent HTTP endpoint. This distinguishes it from sibling tools which are unrelated to GPU listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies this tool is for checking GPU availability, but does not explicitly state when to use it versus alternatives or any prerequisites. However, due to its simplicity (no parameters), usage is largely self-evident.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It correctly implies a read-only operation without side effects. It does not specify edge cases but is adequate for a simple getter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, succinct sentence that front-loads the key action and resource. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with no parameters and an output schema, the description provides all necessary context: it returns the current effective placement. No gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, so schema coverage is 100%. The description adds no parameter-specific value, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Show the current effective multi-GPU model placement.' It uses a specific verb ('Show') and resource ('multi-GPU model placement'), and distinguishes itself from siblings like set_placement and clear_placement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
While no explicit when/when-not guidance is given, the tool's purpose is straightforward and the name implies its usage relative to siblings (set, clear, preview). Context is clear enough for an agent to infer when to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden. It explicitly states that source files are never deleted and explains matching behavior for folders and globs. This covers key behavioral traits, though rate limits or reversibility are not addressed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, no wasted words. The first sentence clearly states purpose, and the second adds critical behavioral context. It is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter and the existence of an output schema, the description covers input semantics and key behavioral notes. It lacks any mention of return values, but the output schema likely provides that. Minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% for the 'names' parameter, but the description thoroughly explains that names can be source names, folder names, or glob patterns, with specific semantics for each. This adds significant meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The verb 'Remove' and resource 'documents from the index' are clear. It specifies the input types (source name, folder, glob pattern) and distinguishes from sibling tools like 'add' and 'sync'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (to remove indexed documents) but does not explicitly state when not to use it or mention alternative tools. No clear guidance on prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes return format (chunks with citations) and scope semantics. No annotations provided, so description carries full burden; lacks details on top_k range or rate limits, but adds useful context beyond schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences plus a scope note; no fluff. Front-loaded purpose and guidelines.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, usage, scope, and output format. Lacks top_k default/range and any mention of permissions, but given output schema exists, it's sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 0% coverage, so description compensates by explaining scope values and default. Query is assumed, top_k is not explained but is self-explanatory. Still, partial coverage justifies 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches indexed documents, code, and crawled pages, distinguishing it from web-fetch or file-read tools. It specifies the verb and resources, with a clear preference note.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly recommends this tool over web-fetch or file-read, and clarifies scope usage with default values and wiki limitation. Missing when-not-to-use conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the output fields (value, default, type, help, choices) and parameter behavior. Despite no annotations, it gives sufficient transparency for safe usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two succinct lines, front-loaded with the main purpose, and each word adds value. No redundancy or waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (single optional parameter, list operation), the description covers output structure and parameter behavior. No missing critical details like authentication or pagination, which are likely not needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter 'group' has zero coverage in schema but the description adds full semantics: filtering behavior, case-insensitivity, and default when empty. This compensates well for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and the resource ('writable lilbee settings'), with specific details about output fields. The name and description distinguish it from siblings like settings_get and settings_reset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the optional group parameter's behavior (case-insensitive filter, empty returns all), providing clear context for when to use it. While it doesn't explicitly contrast with sibling tools, the purpose is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral disclosure burden. It discloses an important trait: it is registered even when the wiki is disabled, similar to an HTTP status route. This gives context about availability and what the tool can reveal, which is sufficient for a status check tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the main purpose front-loaded clearly in the first sentence. The second sentence adds a meaningful caveat about disabled state without unnecessary elaboration. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists, the description need not explain return values. It covers the essential purpose and the notable behavior about disabled state. It could mention what the response looks like, but for a simple status tool this is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero parameters, so schema coverage is trivially 100%. The description correctly omits parameter details. Baseline for a zero-parameter tool is 4, and no additional explanation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Show wiki layer status' with specific contents (page counts, recent lint issues). It distinguishes itself from sibling tools like 'status' by focusing on the wiki layer, and the name reinforces this specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
It explains a key usage context: even when the wiki is disabled, this tool remains registered so callers can read the disabled state. This implies using this tool when you need wiki status regardless of enabled state, but it does not explicitly compare to alternatives like 'status'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses that the tool deletes all generated wiki pages and their indexed rows, and that it requires confirmation (confirm=true). It also explains its availability even when the wiki is disabled, which is valuable behavioral context. With no annotations, this description carries the full burden and does a good job conveying the destructive and conditional nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the action and requirement, the second provides rationale for the tool's availability. Every sentence adds value and it is front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple destructive tool with one parameter, the description covers the action, the confirmation requirement, and a specific use case (even when wiki disabled). It doesn't explain return values or permissions, but an output schema is present, so that's not required. The description is complete enough for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema only defines a boolean 'confirm' with no description, but the description explicitly instructs to 'Pass confirm=true,' clarifying the parameter's role and required value. This adds meaning beyond the schema, though it could further explain the behavior when confirm is false.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete every generated wiki page and its indexed rows.' This is a specific verb (delete), resource (wiki pages and indexed rows), and scope (every). It distinguishes the tool from siblings like 'remove' or 'reset' by focusing on generated wiki pages.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that the tool is 'Registered even with the wiki disabled, because turning the setting off leaves the pages generated earlier in place,' providing clear context for when to use it. It also instructs to 'Pass confirm=true,' indicating the required action. However, it doesn't explicitly mention when not to use it or name alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses that the operation persists to config (indicating state change) and details the spec structure. It does not mention permissions, side effects on other placements, or error handling, but the core behavioral traits are well covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded, stating the purpose in the first sentence. It includes a compact example and additional details in a few sentences without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists, the description does not need to explain return values. It covers the spec format, optional fields, and the persistence behavior. Minor gaps include lack of validation rules or error scenarios, but overall it provides sufficient context for a single-parameter set operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides only an object type with no inner structure, resulting in 0% coverage. The description fully compensates by specifying the expected format: a mapping from role to an object containing devices array and optional tensor_split array, with a concrete JSON example. This adds essential meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Set and apply'), the resource ('manual multi-GPU placement spec'), and the key behavioral trait ('persists to config'). It effectively distinguishes this tool from its siblings like get_placement, preview_placement, and clear_placement by focusing on setting/applying rather than retrieving or clearing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear example of the expected format and explains how to specify roles and devices, including the optional tensor_split. It mentions that omitting a role leaves it auto-placed. However, it does not explicitly state when this tool should be avoided (e.g., use preview_placement for dry-run) or list alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
GitHub Badge
Glama performs regular codebase and documentation scans to:
- Confirm that the MCP server is working as expected.
- Confirm that there are no obvious security issues.
- Evaluate tool definition quality.
Our badge communicates server capabilities, safety, and installation instructions.
Card Badge
Copy to your README.md:
Score Badge
Copy to your README.md:
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/tobocop2/lilbee'
If you have feedback or need assistance with the MCP directory API, please join our Discord server