llm-token-tracker
Server Quality Checklist
Latest release: v1.0.0
- Disambiguation4/5
Most tools have distinct purposes, but get_current_session and get_usage could be confused as both retrieve usage data, though their descriptions differentiate them slightly (session-specific vs. summary). The other tools (clear_usage, compare_costs, get_exchange_rate, track_usage) are clearly distinct.
Naming Consistency5/5All tool names follow a consistent verb_noun pattern with snake_case (e.g., clear_usage, compare_costs, get_current_session). There are no deviations in naming style, making them predictable and readable.
Tool Count5/5With 6 tools, the count is well-scoped for tracking LLM token usage and costs. Each tool serves a specific function in this domain, such as tracking, retrieving, clearing, and comparing data, without being excessive or insufficient.
Completeness3/5The toolset covers core operations like tracking, retrieving, and clearing usage, but lacks update or delete capabilities for specific entries, which might be needed for error correction. The inclusion of get_exchange_rate is useful but slightly tangential to the main purpose.
Average 2.8/5 across 6 of 6 tools scored.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- No commit activity data available
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add
glama.jsonto the root of your repository:{ "$schema": "https://glama.ai/mcp/schemas/server.json", "maintainers": [ "your-github-username" ] }Then . Browse examples.
Add related servers to improve discoverability.
How to sync the server with GitHub?
Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.
To manually sync the server, click the "Sync Server" button in the MCP server admin interface.
How is the quality score calculated?
The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).
Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.
Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).
Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.
Tool Scores
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. 'Get usage summary' implies a read operation, but it doesn't specify if it's safe, requires authentication, has rate limits, or what the output format might be. It's minimal and lacks critical behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with just three words, front-loaded and zero waste. It efficiently states the core action without unnecessary elaboration, though this brevity contributes to gaps in other dimensions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a usage-related tool with no annotations and no output schema, the description is incomplete. It doesn't explain what 'usage' refers to, what data is returned, or how to interpret results, leaving significant gaps for an AI agent to understand and use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage, fully describing the optional 'user_id' parameter. The description adds no additional meaning beyond what the schema provides, such as explaining what 'usage summary' includes or how the parameter affects results. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose2/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get usage summary' restates the tool name 'get_usage' in a slightly different phrasing, making it tautological. It doesn't specify what type of usage (e.g., API, resource, billing) or what 'summary' entails, leaving the purpose vague beyond the obvious from the name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'track_usage' or 'compare_costs' among the sibling tools. The description lacks context about prerequisites, timing, or specific use cases, offering no help in tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Clear usage data' implies a destructive mutation, but it doesn't disclose behavioral traits like whether this is irreversible, requires admin permissions, affects other data, or has side effects. The description is minimal and lacks critical context for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description 'Clear usage data' is extremely concise with zero waste, using only three words. However, it's arguably under-specified rather than optimally concise, as it lacks necessary detail for a mutation tool. It's front-loaded but too brief to be fully helpful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (a destructive mutation with one parameter), lack of annotations, and no output schema, the description is incomplete. It doesn't cover behavioral aspects, return values, or error conditions, leaving significant gaps for an agent to understand and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the parameter 'user_id' documented as 'User ID to clear'. The description adds no meaning beyond this, as it doesn't explain parameter usage, format, or constraints. With high schema coverage, the baseline score of 3 is appropriate, as the schema handles parameter documentation adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Clear usage data' states a verb ('Clear') and resource ('usage data'), providing a basic purpose. However, it's vague about what 'clear' means (delete, reset, archive?) and doesn't distinguish from sibling tools like 'get_usage' or 'track_usage'. It's not tautological but lacks specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites, context, or exclusions, and there's no reference to sibling tools like 'get_usage' for comparison. Usage is implied only by the tool name and description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. The description only states what the tool does at a high level ('compare costs') without explaining how it behaves: whether it requires authentication, what data sources it uses, whether it makes network calls, what format the comparison results take, or any rate limits. This leaves significant gaps in understanding the tool's operational characteristics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at just three words: 'Compare costs between models'. It's front-loaded with the core action and contains no unnecessary information. Every word serves a purpose in conveying the tool's function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's apparent complexity (comparing costs between models likely involves multiple data sources and calculations), the description is insufficient. With no annotations, no output schema, and only a minimal description, there's inadequate information about what the tool returns, how it performs comparisons, or what models are involved. The description doesn't compensate for the lack of structured metadata.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the single parameter 'tokens' clearly documented as 'Number of tokens to compare'. The description doesn't add any additional meaning beyond what the schema provides—it doesn't explain what 'tokens' represent in context or how they relate to cost comparison. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Compare costs between models' states a general purpose but lacks specificity. It mentions the verb 'compare' and resource 'costs between models', but doesn't specify what models are being compared or how the comparison works. It doesn't distinguish from sibling tools like 'get_exchange_rate' or 'track_usage' which might involve cost-related operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There are no explicit instructions about when to invoke it, when not to use it, or what alternatives exist among the sibling tools. The user must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions 'intuitive format' but doesn't disclose behavioral traits such as whether this is a read-only operation, if it requires authentication, rate limits, or how it handles errors. The description is too vague to provide adequate transparency for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose. However, it could be more structured by separating the purpose from the format details, but it avoids unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a tool that likely returns usage data, the description is incomplete. It doesn't explain the return format beyond 'intuitive format', lacks details on what 'cost' refers to, and doesn't cover potential errors or edge cases, leaving significant gaps for an agent to understand the tool fully.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters (user_id and total_budget) with descriptions and defaults. The description adds no additional meaning beyond what the schema provides, such as explaining how these parameters affect the output or their practical use cases.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get current session usage' with specific metrics (remaining, used, input/output tokens, cost). It uses a specific verb ('Get') and resource ('current session usage'), though it doesn't explicitly distinguish from sibling tools like 'get_usage' or 'track_usage'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_usage' or 'track_usage'. It mentions an 'intuitive format' but doesn't specify contexts or exclusions for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Track' implies a write or logging operation, but the description doesn't disclose whether this creates records, updates a database, requires authentication, has side effects, or returns any confirmation. For a mutation-like tool with zero annotation coverage, this is a significant gap in behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without any wasted words. It's appropriately sized and front-loaded, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (a tool with 5 parameters, no annotations, and no output schema), the description is incomplete. It doesn't explain what 'track' entails operationally, what happens after invocation, or how this differs from sibling tools. For a tool that likely modifies state, more context is needed to guide proper usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 5 parameters. The description adds no additional meaning beyond implying these parameters are used for tracking token usage, which is already evident from the schema. This meets the baseline of 3 when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('track') and resource ('token usage for an AI API call'), making the purpose immediately understandable. However, it doesn't differentiate this tool from sibling tools like 'get_usage' or 'clear_usage', which likely handle related aspects of usage data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'get_usage' (likely for retrieval) and 'clear_usage' (likely for deletion), the agent has no indication whether this is for logging, monitoring, or another purpose, or what prerequisites might exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
- Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions 'cache info', which hints at caching behavior, but doesn't disclose details like cache duration, refresh mechanisms, or rate limits. It implies a read operation but doesn't specify error conditions or authentication needs. The description adds some context but lacks comprehensive behavioral details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Get current USD to KRW exchange rate') and adds a key feature ('with cache info') without unnecessary words. It's appropriately sized for a simple tool with one parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (one optional parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and hints at caching behavior, but lacks details on return values, error handling, or integration context. For a financial data tool, more completeness would be beneficial, but it meets the minimum for this context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with one parameter ('force_refresh') fully documented in the schema. The description doesn't add any parameter-specific information beyond what the schema provides, such as explaining when to use 'force_refresh' or its impact on caching. With high schema coverage, the baseline is 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Get current USD to KRW exchange rate with cache info'. It specifies the verb ('Get'), resource ('USD to KRW exchange rate'), and an additional feature ('cache info'). However, it doesn't explicitly differentiate from sibling tools, which appear unrelated (usage tracking, session management, cost comparison).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention any prerequisites, constraints, or scenarios where this tool is preferred over other methods. The sibling tools seem unrelated, but no explicit comparison or exclusion criteria are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
GitHub Badge
Glama performs regular codebase and documentation scans to:
- Confirm that the MCP server is working as expected.
- Confirm that there are no obvious security issues.
- Evaluate tool definition quality.
Our badge communicates server capabilities, safety, and installation instructions.
Card Badge
Copy to your README.md:
Score Badge
Copy to your README.md:
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/wn01011/llm-token-tracker'
If you have feedback or need assistance with the MCP directory API, please join our Discord server