similarity_score
Compute text similarity between reference and hypothesis using multiple metrics: Cosine (BoW, TF-IDF), Jaccard, ROUGE-1, ROUGE-2, ROUGE-L, and BLEU. No API key needed. Ideal for LLM eval (expected vs actual), RAG quality checks, and NLG benchmarking. Supports batch mode.
Example tool call
This call uses example arguments supplied in the tool’s input schema.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"reference": "the cat sat on the mat",
"threshold": 0.5,
"hypothesis": "the cat sat on the mat"
},
"name": "similarity_score"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| batch | No | Batch mode: array of {reference, hypothesis} pairs. | |
| metrics | No | Metrics to compute (default: all). Options: "cosine_bow", "cosine_tfidf", "jaccard", "rouge1", "rouge2", "rougeL", "bleu" | |
| reference | No | Reference / expected text (ground truth) | |
| threshold | No | Optional pass/fail threshold (0-1). Applies to ROUGE-L F1 score. | |
| hypothesis | No | Hypothesis / actual text (LLM output) |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| f1 | No | ||
| mode | No | ||
| count | No | ||
| recall | No | ||
| results | No | ||
| precision | No |