addedInput schema / examples
Added value: +[
+ {
+ "base": {
+ "output": "Paris is the capital and largest city of France."
+ },
+ "variants": [
+ {
+ "output": "Paris is the capital and largest city of France.",
+ "relation": "case"
+ }
+ ]
+ }
+]
changedInput schema / properties / api_key / description
Previous value: -"OpenAI API key — required only when mode is embeddings."New value: +"OpenAI key, embeddings mode only."
changedInput schema / properties / base / description
Previous value: -"The reference run: the original question and the answer your system produced for it."New value: +"The original run."
changedInput schema / properties / base / properties / output / description
Previous value: -"Required — the answer your system produced for the original question."New value: +"The answer to the original question."
changedInput schema / properties / base / properties / question / description
Previous value: -"Optional — the original question, echoed back in the output for readability."New value: +"Optional, echoed back."
changedInput schema / properties / baseline_guard / description
Previous value: -"Correctness floor applied to the BASE answer before anything is scored. Failing it returns INVALID, not FAIL."New value: +"Floor on the BASE answer; failing it returns INVALID."
changedInput schema / properties / baseline_guard / properties / min_length / description
Previous value: -"Minimum base answer length in chars (default 40)."New value: +"Default 40 chars."
changedInput schema / properties / baseline_guard / properties / must_not_match / description
Previous value: -"Refusal patterns, matched case-insensitively in the FIRST 200 CHARS of the base answer (refusals lead; matching anywhere would flag a long correct answer that merely mentions one). Replaces the FR+EN default list, never merges — pass [] to disable."New value: +"Refusal patterns, checked in the first 200 chars. Replaces the default FR+EN list; [] disables."
changedInput schema / properties / mode / description
Previous value: -"tfidf (default): free, lexical, deterministic — but a genuine paraphrase rarely reaches 0.80, so gate on case/typo and treat paraphrase as a trend. embeddings: OpenAI text-embedding-3-small, true semantic similarity, requires api_key. translation requires this mode."New value: +"tfidf (default, lexical: gate on case/typo, read paraphrase as a trend) or embeddings (OpenAI, needs api_key; required for translation)."
changedInput schema / properties / require_all / description
Previous value: -"If true (default), every gated variant must pass. KEEP THE DEFAULT for any run you gate on. Setting it false is not a tolerance dial but an off switch: relations have asymmetric pass rates (a typo variant usually scores ~1.0 because the answer really is identical), so one trivial row is enough to hold the whole run at PASS while a paraphrase fails. When that happens the result carries an explicit warning naming the failed rows."New value: +"Default true: every gated variant must pass. Keep it for a gate — false lets one easy typo row hold the run at PASS while a paraphrase fails."
changedInput schema / properties / thresholds / description
Previous value: -"Per-relation threshold overrides. Calibrate on your own corpus before gating — the defaults are starting points, not measurements."New value: +"Per-relation overrides, 0-1. Calibrate on your own data before gating."
changedInput schema / properties / variants / description
Previous value: -"Answers produced for transformed versions of the same question, each tagged with the relation that was applied."New value: +"Answers to transformed versions of the question."
removedInput schema / properties / variants / items / properties / id / description
Removed value: -"Optional identifier (default: variant_<n>)."
changedInput schema / properties / variants / items / properties / output / description
Previous value: -"Required — the answer your system produced for the transformed question."New value: +"The answer to the transformed question."
changedInput schema / properties / variants / items / properties / question / description
Previous value: -"Optional — the transformed question, echoed back for readability."New value: +"Optional, echoed back."
changedInput schema / properties / variants / items / properties / relation / description
Previous value: -"Transformation applied to the QUESTION. Decides the metric and the default threshold."New value: +"Transformation applied to the question; sets the metric and threshold."