Skip to main content
Glama

x402-tokenize

Tokenize: Split text into word tokens. Provide text or input; returns the array of whitespace-separated words.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
textNoText to process
inputNoInput to process
lowercaseNoLowercase to process

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed3 schema fields changed
    • addedInput schema / properties / input
      Added value: +{
      +  "description": "Input to process",
      +  "type": "string"
      +}
    • addedInput schema / properties / lowercase
      Added value: +{
      +  "description": "Lowercase to process",
      +  "type": "string"
      +}
    • addedInput schema / properties / text
      Added value: +{
      +  "description": "Text to process",
      +  "type": "string"
      +}
  2. First observed

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden, and it does disclose the return value (an array of whitespace-separated words), which is genuinely useful. It does not say how punctuation, empty input, or multiple whitespace runs are handled, so behavioral coverage is only partial for a pure-function tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short clauses, front-loaded with the verb and resource, then the input convention, then the return shape. Nothing is wasted, though the sentence is slightly compressed to the point of ambiguity about the `text`/`input` choice.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so describing the return array is the right move and it is done. But with three optional, overlapping string parameters and zero required parameters, the description omits the decisive details (which input wins, what `lowercase` actually does), leaving an agent under-equipped for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. However the schema descriptions are generic ("Text to process", "Input to process", "Lowercase to process") and the `lowercase` parameter is typed as a string with no clue as to intent; the description says "provide text or input" but never resolves the overlap between the two string inputs or the odd typing of `lowercase`.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb (split/tokenize) and resource (text into word tokens) and states the return shape. It is immediately understandable, but it offers no differentiation from the many sibling text tools (x402-chunk, x402-word-count, x402-sentence-split, x402-ngram), so an agent must infer where tokenize fits among them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Provide text or input" is the only usage hint, and it never says when to prefer this tool over sibling splitting/n-gram tools. There is no statement of exclusions, prerequisites, or alternatives, leaving selection to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources