Evaluate the quality of a translation using xCOMET model.

This tool analyzes a source text and its translation, providing:

• A quality score between 0 and 1 (higher is better)

• Detected error spans with severity levels (minor/major/critical)

• A human-readable quality summary

Args:

• source (string): Original source text to translate from

• translation (string): Translated text to evaluate

• reference (string, optional): Reference translation for comparison

• source_lang (string, optional): Source language code (ISO 639-1)

• target_lang (string, optional): Target language code (ISO 639-1)

• response_format ('json' | 'markdown'): Output format (default: 'json')

• use_gpu (boolean, optional): Use GPU for inference if available (default: false)

Returns: For JSON format: { "score": number, // Quality score 0-1 "errors": [ // Detected errors { "text": string, "start": number, "end": number, "severity": "minor" | "major" | "critical" } ], "summary": string // Human-readable summary }

Examples:

• Evaluate EN→JA translation quality

• Check if MT output needs post-editing

• Compare translation against reference

Detect and categorize errors in a translation.

This tool focuses on error detection, providing detailed information about translation errors with their severity levels and positions.

Args:

• source (string): Original source text

• translation (string): Translated text to analyze

• reference (string, optional): Reference translation

• min_severity ('minor' | 'major' | 'critical'): Minimum severity to report (default: 'minor')

• response_format ('json' | 'markdown'): Output format (default: 'json')

• use_gpu (boolean, optional): Use GPU for inference if available (default: false)

Returns: { "total_errors": number, "errors_by_severity": { "minor": number, "major": number, "critical": number }, "errors": [ { "text": string, "start": number, "end": number, "severity": "minor" | "major" | "critical", "suggestion": string | null } ] }

Examples:

• Find critical errors before publication

• Identify areas needing post-editing

• Quality gate for MT output

Evaluate multiple translation pairs in a batch.

This tool processes multiple source-translation pairs and provides aggregate statistics along with individual results.

Args:

• pairs (array): Array of translation pairs, each with:

  • source (string): Original source text

  • translation (string): Translated text

  • reference (string, optional): Reference translation

• source_lang (string, optional): Source language code

• target_lang (string, optional): Target language code

• response_format ('json' | 'markdown'): Output format (default: 'json')

• use_gpu (boolean, optional): Use GPU for inference if available (default: false)

• batch_size (number, optional): Inference batch size, 1-64 (default: 8). Larger = faster but uses more memory.

Returns: { "average_score": number, "total_pairs": number, "results": [ { "index": number, "score": number, "error_count": number, "has_critical_errors": boolean } ], "summary": string }

Examples:

• Evaluate entire translated document

• Compare MT system quality across test set

• Identify segments needing attention