Supports using OpenAI models as independent raters for multi-model evaluation and statistical consensus measurement alongside other AI providers.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
Multi-model rating for AI evaluation: measures agreement among model raters (Fleiss' κ, Kendall's W) to find contested items for expert review.
What agreement tells you: where your raters and your criteria are consistent, and where they are not — which items are contested, and whether disagreement comes from the criteria or the raters.
What it does not tell you: that the raters are right. Agreement is reliability, not validity. Models can agree confidently on a wrong answer. Use ConKurrence to decide where an expert's judgment is needed and to diagnose your evaluation criteria — not to replace expert-labelled ground truth.
Status: maintenance mode. ConKurrence is developed by JE Vectors LLC as the evaluation instrument behind Inqura. It is not accepting purchases and carries no support commitment.
npm install -g conkurrenceRelated MCP server: CI-1T Prediction Stability Engine
Use ConKurrence as an MCP server in Claude Desktop or any MCP-compatible client:
npx conkurrence mcpAdd to your claude_desktop_config.json:
{
"mcpServers": {
"conkurrence": {
"command": "npx",
"args": ["-y", "conkurrence", "mcp"]
}
}
}/plugin marketplace add AlligatorC0der/conkurrenceMulti-model evaluation — Run your schema against Bedrock, OpenAI, and Gemini models simultaneously
Agreement statistics — Fleiss' kappa with bootstrap confidence intervals; Kendall's W against expert-labelled anchor items
Self-consistency mode — No API keys needed; uses the host model via MCP Sampling
Schema suggestion — AI-powered schema design from your data
Trend tracking — Compare runs over time, detect agreement degradation
Cost estimation — Know the cost before running
Tool | Description |
| Execute an evaluation across multiple AI raters |
| Generate a detailed markdown report |
| Side-by-side comparison of two runs |
| Track agreement over multiple runs |
| AI-powered schema suggestion from your data |
| Validate a schema before running |
| Estimate cost and token usage |
Homepage: conkurrence.com
Terms of Service: app.conkurrence.com/terms
Privacy Policy: app.conkurrence.com/privacy
BUSL-1.1 — Business Source License 1.1
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
LLM & RAG evals: hallucination, eval contracts, semantic tests, tokens & model info. Free, no key.
Checks if a cheaper model agrees with your GPT, Claude or Gemini outputs, and what routing saves.
Evaluate agent workflows and LLMs on your own data: find wasted steps, pick models, catch drift.
MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
