Skip to main content
Glama
Sign In
EvalForge Lite

EvalForge Lite

by thejaredchapman
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

Related Servers

Alternatives to EvalForge Lite

No user-submitted related servers found.

    Related Servers

    • AI Model Experiments

      pipeworx-io
      A
      license
      Not graded
      quality
      C
      maintenance
      Enables running the same prompt(s) across roughly 300 models from Anthropic, OpenAI, Google, Meta, Mistral, DeepSeek, Qwen and others, then comparing per-model outputs, latency, token usage, error rates and billed cost side by side. Supports free cost estimates before launching, asynchronous run creation with status polling and cancellation, prepaid x402 credit top-ups, and an optional AI-written comparison summary of the results.
      126 npm
      MIT

    • BlindWrite MCP

      ygstudio-game
      A
      license
      A
      quality
      B
      maintenance
      Enables blind, bias-free AI writing evaluations through randomized A/B duels, Bradley-Terry MLE rankings, and preference analytics, while letting Claude delegate heavy drafting to OpenRouter models to save tokens.
      11
      MIT

    • llm-eval-harness

      razilzul
      A
      license
      Not graded
      quality
      C
      maintenance
      Enables coding agents to evaluate a single model over a golden set with trap cases, reporting accuracy overall and on traps, mean latency, cost per request, and failing cases. Also compares a baseline against a candidate model and returns a pass/fail gate verdict with failure reasons, so a model swap can be checked before it ships.
      MIT

    • brandcheck

      CloudCorpRecords
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables starting and polling hosted visibility measurements that ask real buyer questions to OpenAI, Anthropic and Gemini, returning a verdict, question-win counts, named competitors, and which providers answered. Requires no API key or signup, with results forwarded verbatim from the upstream service.
      309 npm
      MIT

    • galton

      Luissalet
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables assistants to queue and monitor local language and vision model evaluations, compare results with confidence intervals, and retrieve routing recommendations for task categories.
      MIT
    Deploy Server
    Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

    TDQS

    A3.7/5.0

    Scored across 9 tools

    Disambiguation4/5

    Most tools have clearly distinct purposes, but the three model-introspection tools (list_models, suggest_models, list_availability) overlap somewhat, and evaluate_prompt vs run_comparison could be momentarily confused since both invoke LLMs. Descriptions do help differentiate them, especially the explicit pre-run vs full-run distinction.

    Naming Consistency5/5

    All tools use snake_case with a consistent verb_noun pattern (list_models, suggest_models, set_policy, evaluate_prompt, run_comparison, list_runs, get_report). Even the format variant get_report_csv follows the same convention cleanly.

    Tool Count5/5

    Nine tools is well-scoped for a model-evaluation server, with each tool earning its place across catalog introspection, policy, evaluation, execution, and reporting. No padding or redundancy in count.

    Completeness4/5

    Coverage spans discovery, policy, prompt feedback, comparison runs, and report exports (PDF/CSV), which is solid for the domain. Minor gaps: no structured JSON run-detail retrieval or run/policy deletion, and set_policy has no corresponding get_policy to inspect current state.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Resources