lsh_dedup
Detect and remove duplicate texts using SimHash, MinHash, and Hamming distance comparisons, with per-namespace dedup fences to cut redundant calls.
Instructions
[Cost: $0.0005 USDC on Base & Solana] Sub-millisecond LSH deduplication oracle.
Compute 64-bit SimHash fingerprints, estimate Jaccard similarity via MinHash,
compare texts by Hamming distance, and maintain per-namespace dedup fences to
eliminate redundant agent tool calls and duplicate web scraping payloads.
Actions:
simhash — Compute 64-bit SimHash fingerprint(s).
Params: {"text": "..."} or {"texts": ["...", "..."]}
compare — Hamming distance between two fingerprints or texts.
Params: {"text_a": "...", "text_b": "..."} or
{"fingerprint_a": 12345, "fingerprint_b": 67890}
minhash — Jaccard similarity via MinHash signature comparison.
Params: {"text_a": "...", "text_b": "...", "threshold": 0.8}
dedup — Check text against seen-set fence; add if new.
Params: {"text": "...", "namespace": "my_swarm", "threshold": 3}
stats — Namespace statistics.
Params: {} or {"namespace": "my_swarm"}
clear — Reset namespace or all namespaces.
Params: {} or {"namespace": "my_swarm"}
Args:
action: Sub-action to execute (simhash/compare/minhash/dedup/stats/clear).
params: Action-specific parameter dictionary.
payment_signature: x402 Base USDC transaction hash (defaults to dev mock key).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| action | No | simhash | |
| params | No | ||
| payment_signature | No |