RemoveDuplicates.org
Server Details
Remove duplicate lines or CSV/TSV rows from text. Stateless: processed per call, never stored.
- Status
- Healthy
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
- Repository
- describesomeone/remove-duplicates
- GitHub Stars
- 1
TDQS
Scored across 1 tool
There is only one tool, so there is no possibility of confusing it with another tool. The agent will always know exactly which tool to call for this server's single purpose.
The single tool uses a clear snake_case verb_noun format (remove_duplicates). With only one tool, naming inconsistency cannot occur.
One tool is slightly thin for a full-featured service, but given the narrowly focused purpose of RemoveDuplicates.org, a single deduplication tool is reasonable. The tool handles both line lists and CSV/TSV data, broadening its usefulness.
The tool covers the core deduplication use cases for plain text, CSV, and TSV input, and returns cleaned output with counts. More advanced options such as case-sensitive/insensitive matching or column-specific deduplication are not exposed, but are not required for the stated primary function.
Available Tools
1 toolremove_duplicatesRemove duplicatesARead-onlyIdempotentInspect
Remove duplicate lines from a text list, or duplicate rows from CSV/TSV text such as rows copied from Excel or Google Sheets. Uses the same engine as removeduplicates.org and returns the cleaned text with counts. Stateless: the text is processed in memory for this call only and is never stored.
| Name | Required | Description | Default |
|---|---|---|---|
| keep | No | Which occurrence of a duplicate survives. | first |
| text | Yes | The list or table text: one item per line, or CSV/tab-separated rows. Up to 131072 UTF-8 bytes. | |
| trim | No | Ignore surrounding spaces when comparing and return trimmed values. | |
| order | No | preserve keeps the original order; sort returns a natural (numeric-aware) sort. | preserve |
| format | No | auto detects CSV/TSV tables; lines compares whole lines; table forces CSV/TSV parsing. | auto |
| header | No | Tables only: keep the first row as a header outside duplicate checks and counts. | |
| compare | No | Tables only: "row" compares entire rows; a zero-based column index or an array of indices compares only those columns. | row |
| ignoreCase | No | Treat values such as Apple and apple as duplicates. | |
| removeEmpty | No | Drop blank lines or all-blank rows. |
Output Schema
| Name | Required | Description |
|---|---|---|
| text | Yes | The cleaned text, rows joined with \n. |
| stats | Yes | |
| format | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds meaningful behavior beyond these: it is stateless ('processed in memory for this call only and is never stored') and returns 'the cleaned text with counts.' This gives the agent insight into execution and output without contradicting any annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences long, each serving a distinct purpose: the core action, the engine and output, and the statelessness guarantee. It is front-loaded with the primary function and contains no redundant or extraneous text, making it highly efficient and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a transformation tool with a rich schema and an output schema (indicated by context), the description covers all necessary invocation context: input types, output (cleaned text with counts), and statelessness. It does not need to detail parameter behavior since the schema does that. The only minor gap is lack of explicit edge-case examples, but this is not critical given the schema's thoroughness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides 100% coverage, with detailed descriptions for all 9 parameters, including enums, defaults, and semantics. The tool description adds no extra parameter guidance—it only restates the general purpose. Since the schema fully documents parameters, a baseline score of 3 is appropriate; the description does not need to compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear, specific verb and resource: 'Remove duplicate lines from a text list, or duplicate rows from CSV/TSV text.' It also names the engine and mentions the stateless nature, leaving no ambiguity about what the tool does. Since there are no sibling tools, differentiation is unnecessary, and the scope is explicitly defined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly indicates when to use the tool: for text lists or table data such as rows from Excel/Google Sheets. It provides concrete input examples and states the stateless behavior. While it doesn't explicitly say when not to use it, the absence of siblings makes this less critical; the context is sufficiently clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
1 tool update
- First observed
remove_duplicates
Related MCP Connectors
Dedupe, flatten and clean messy JSON rows (emails, phones, URLs, HTML) in one call, as JSON or CSV.
Normalize messy CSV into your schema; ambiguous values return as exceptions. $0.02 USDC via x402.
Compare CSV tables by key; report changes and ambiguous rows. 0.05 USDC on Base via x402.
Clean, filter and reshape JSON rows in one call: 26 transforms, 25 filters, sort, dedupe, limit.
Related MCP Servers
- FlicenseAqualityDmaintenanceEnables LLM agents to find and safely delete duplicate files using content-based detection, with smart copy prioritization and dry-run mode.3-
- FlicenseNot gradedqualityDmaintenanceManages LinkedIn prospecting CSV files with deduplication and search, reducing LLM token usage by offloading file operations.-
- AlicenseAqualityCmaintenanceEnables validating CSV structure, checking simple schemas, converting between CSV and JSON, sampling rows, and finding duplicate keys on local files. All processing stays local, so no user data is ever uploaded.6MIT
- AlicenseAqualityBmaintenanceCompresses verbose MCP output by deduplicating repeated lines and summarizing JSON arrays, achieving 60-80% token savings for LLM prompts.1MIT
Glama MCP Gateway
Add one secure layer between your agents and this server.