What Woodstitch measures: the two templates (tts_preference.v1: two clips, forced choice; tts_detect.v1: one clip, human or AI + confidence), their inputs and metrics, and what is NOT offered (MOS, CMOS, MUSHRA, any 1-5 scale). No API key needed.

Rate card: USD per hour of audio a listener hears, priced per judgement from the measured duration of your clips, with worked examples and limits. No API key needed.

Account balance and spend for the configured API key.

Your last 50 evaluations with status and progress.

Upload audio files from the local disk (wav, mp3, flac, ogg, m4a; 0.5-120 s each, 50 MB per file, up to 100 files per call). Returns file ids, measured durations and formats, in the order sent. Use an id anywhere woodstitch_create_eval takes an audio reference. Re-uploading the same bytes is harmless (content-addressed).

List your uploaded files, newest first.

Delete one upload from your account by file id. Studies that already used the file keep their copy.

Create and PRICE an evaluation. Charges nothing. Resolves every clip (file id or public URL), validates it, measures duration and returns a quote with per-pair prices, warnings and the compiled design. Set dry_run=true to price without storing anything (no eval_id). Preference: pass pairs as [{id, a, b, system_a?, system_b?, passage?, speaker?}] where a/b are file ids or URLs rendering the SAME text; sides are shown to listeners in random order. Detection: pass clips as [{id, audio, truth: 'human'|'ai', source, speaker, passage, duration_bucket?}] and include real human recordings of the same passages as the baseline. per_participant = {max_judgements, unique_by: [...], max_per: {...}} limits what one listener hears; for a multi-system comparison use unique_by ['passage'] and max_per {'system': N}. ALWAYS show the returned quote total to the user before starting.

START an evaluation. THIS SPENDS REAL MONEY and cannot be undone once listeners begin. Only call it after the user has seen the quote total returned by woodstitch_create_eval and explicitly agreed. Pass that total as user_confirmed_amount_usd: it is sent to the API as the hard spending ceiling (max_total_usd), so if the current quote is higher than what the user confirmed the API refuses with over_max and nothing starts. Billing is metered from the first judgement; a study pauses rather than overspends if the balance runs out.

Progress and live score for an evaluation: status, judgements done/target, pairs with data, provisional metrics, dashboard_url, and max_additional_charge_if_canceled_now_usd. Cheap to poll. Humans take minutes to days; do not sit in a tight loop. Interim numbers are provisional.

Final numbers once the evaluation is completed (or canceled, then incomplete=true). Preference: human_pref/b_win_rate is the share of preference for your b side with a 95% CI; 0.5 = indistinguishable. Detection: human_label_rate per source with two intervals: tested_stimuli (these files) and model (generalises over voices and passages; 3-4x wider; quote THIS one for any claim about a model). Returns not_ready while still running.

Every judgement as a row (pseudonymous rater, pair/clip id, source, speaker, passage, answer, confidence, time spent). format 'json' returns the rows; 'csv' returns CSV text. Pass save_to (a local path) to write the export to disk instead of returning it, which is better for large studies.

Stop an evaluation early. Charges only for work already handed to listeners; anyone mid-task finishes and is paid. Check max_additional_charge_if_canceled_now_usd via woodstitch_get_eval first if the user wants that number. Requires confirm=true.