deduplicate_strings

Remove duplicate strings and select a diverse subset using semantic embeddings and optimization. Ideal for reducing redundancy and identifying representative content from large text collections.

Instructions

Get top-k semantically unique strings from a list using Jina embeddings and submodular optimization. Use this when you have many similar strings and want to select the most diverse subset that covers the semantic space. Perfect for removing duplicates, selecting representative samples, or finding diverse content.

Input Schema

Name	Required	Description	Default
`k`	No	Number of unique strings to return. If not provided, automatically finds optimal k by looking at diminishing return
`strings`	Yes	Array of strings to deduplicate

Input Schema (JSON Schema)

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "additionalProperties": false,
  "properties": {
    "k": {
      "description": "Number of unique strings to return. If not provided, automatically finds optimal k by looking at diminishing return",
      "type": "number"
    },
    "strings": {
      "description": "Array of strings to deduplicate",
      "items": {
        "type": "string"
      },
      "type": "array"
    }
  },
  "required": [
    "strings"
  ],
  "type": "object"
}

Jina AI Remote MCP Server