Skip to main content
Glama

Text similarity and near-duplicates

ai_text_similarity
Read-onlyIdempotent

Compare two or more texts or files to measure wording overlap with Jaccard, MinHash, and cosine scores; rank similar pairs or group near-duplicates using a threshold.

Instructions

Compare two or more texts (or text files) for overlap. For every pair returns: Jaccard similarity of word shingles (k=3 words by default), a MinHash estimate of that Jaccard (128 hashes), and cosine similarity of word frequencies. Words are lower-cased, punctuation and Arabic diacritics ignored. Measures wording overlap, not meaning (no embeddings): paraphrases score low. Mode "pairs" (default) lists pairs sorted by the chosen metric; mode "dedupe" groups near-duplicates whose metric is at least "threshold" (default 0.8; groups are connected, so AB and BC put A, B, C together). Give "texts" (inline strings) and/or "paths" (file paths or base64). Up to 1000 texts.

Input Schema

NameRequiredDescriptionDefault
modeNo"pairs" (similarity of each pair) or "dedupe" (groups of near-duplicates).pairs
pathsNoOr/also: UTF-8 text files (local paths or base64).
textsNoTexts to compare (inline).
labelsNoOptional names for the texts, in order (texts first, then paths). Default: "#1", "#2"… or the file name.
metricNoMetric used to sort pairs and to group duplicates. Default jaccard.jaccard
maxPairsNoMost pairs to return (highest first). Default 200.
numHashesNoMinHash signature length. Default 128 (error about ±0.09).
thresholdNoDedupe: pairs at or above this are duplicates. Default 0.8.
shingleSizeNoWords per shingle (k). Default 3.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only cover read-only/idempotent/local, but the description adds real behavioral detail: normalization rules (lowercased, punctuation and Arabic diacritics ignored), the fact that dedupe groups are connected components (A~B, B~C merge), the 1000-text cap, and the metric-selection effect on both sorting and grouping.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the purpose, then algorithms, then semantics caveat, then mode/input mechanics. Dense but every sentence carries a distinct fact; no restatement of the title or name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 9 params, no required fields, and no output schema, the description still tells the agent what each pair returns and how results are ordered/grouped. An agent can pick a mode, choose a metric, and set a threshold correctly without further probing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3; the description goes further by tying 'threshold' to connected grouping semantics, noting shingle size k=3 as words per shingle, and clarifying that 'texts' and 'paths' can be combined. It does not explain 'labels' ordering or 'numHashes' accuracy tradeoff (that lives in the schema), so short of a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Compare two or more texts for overlap') and immediately enumerates the three returned measures. The caveat 'Measures wording overlap, not meaning (no embeddings): paraphrases score low' distinguishes it from semantic/diff siblings without naming them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly explains the two modes and when each applies ('pairs' default lists sorted pairs; 'dedupe' groups near-duplicates at or above threshold), plus how to supply input via 'texts' and/or 'paths'. It never names a competing sibling (e.g. text_diff) or says when not to use it, so no full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools