Skip to main content
Glama

AgentDesk MCP — Adversarial AI Review

npm version npm downloads License: MIT Tests MCP

Quality control for AI pipelines — one MCP tool. Works with Claude Code, Claude Desktop, and any MCP client.

29.5% of teams do NO evaluation of AI outputs. (LangChain Survey) Knowledge workers spend 4.3 hours/week fact-checking AI outputs. (Microsoft 2025)

AgentDesk MCP fixes this. Add independent adversarial review to any AI pipeline in 30 seconds.

Quick Start

npx @ezark-publish/agentdesk-mcp

Claude Code

claude mcp add agentdesk-mcp -- npx @ezark-publish/agentdesk-mcp

Claude Desktop

{
  "mcpServers": {
    "agentdesk-mcp": {
      "command": "npx",
      "args": ["-y", "@ezark-publish/agentdesk-mcp"],
      "env": { "ANTHROPIC_API_KEY": "sk-ant-..." }
    }
  }
}

HTTP Transport (Streamable HTTP)

Run as an HTTP server for remote access, Smithery hosting, or multi-client setups:

# Start with HTTP transport on port 3100
MCP_HTTP_PORT=3100 npx @ezark-publish/agentdesk-mcp

# Or use the --http flag (defaults to port 3100)
npx @ezark-publish/agentdesk-mcp --http

MCP endpoint: POST http://localhost:3100/mcp Health check: GET http://localhost:3100/health

Install from GitHub (alternative)

npm install github:Rih0z/agentdesk-mcp

Requirements

  • ANTHROPIC_API_KEY environment variable (uses your own key — BYOK)

Related MCP server: open-code-review

Tools

review_output

Adversarial quality review of any AI-generated output. An independent reviewer assumes the author made mistakes and actively looks for problems.

Input:

Parameter

Required

Description

output

Yes

The AI-generated output to review

criteria

No

Custom review criteria

review_type

No

Category: code, content, factual, translation, etc.

model

No

Reviewer model (default: claude-sonnet-4-6)

Output:

{
  "verdict": "PASS | FAIL | CONDITIONAL_PASS",
  "score": 82,
  "issues": [
    {
      "severity": "high",
      "category": "accuracy",
      "description": "Claim about X is unsupported",
      "suggestion": "Add citation or remove claim"
    }
  ],
  "checklist": [
    {
      "item": "Factual accuracy",
      "status": "pass",
      "evidence": "All statistics match cited sources"
    }
  ],
  "summary": "Overall assessment...",
  "reviewer_model": "claude-sonnet-4-6"
}

review_dual

Dual adversarial review — two independent reviewers assess the output from different angles, then a merge agent combines findings.

  • If either reviewer finds a critical issue → merged verdict is FAIL

  • Takes the lower score

  • Combines and deduplicates all issues

Use for high-stakes outputs where quality is critical.

Same parameters as review_output.

How It Works

  1. Adversarial prompting: The reviewer is instructed to assume mistakes were made. No benefit of the doubt.

  2. Evidence-based checklist: Every PASS item requires specific evidence. Items without evidence are automatically downgraded to FAIL.

  3. Anti-gaming validation: If >30% of checklist items lack evidence, the entire review is forced to FAIL with a capped score of 50.

  4. Structured output: Verdict + numeric score + categorized issues + checklist (not just "looks good").

Use Cases

  • Code review: Check for bugs, security issues, performance problems

  • Content review: Verify accuracy, readability, SEO, audience fit

  • Factual verification: Validate claims in AI-generated text

  • Translation quality: Check accuracy and naturalness

  • Data extraction: Verify completeness and correctness

  • Any AI output: Summaries, reports, proposals, emails, etc.

Why Not Just Ask the Same AI to Review?

Self-review has systematic leniency bias. An LLM reviewing its own output shares the same blind spots that created the errors. Research shows models are 34% more likely to use confident language when hallucinating.

AgentDesk uses a separate reviewer invocation with adversarial prompting — fundamentally different from self-review.

Comparison

Feature

AgentDesk MCP

Manual prompt

Braintrust

DeepEval

One-tool setup

Yes

No

No

No

Adversarial review

Yes

DIY

No

No

Dual reviewer

Yes

DIY

No

No

Anti-gaming validation

Yes

No

No

No

No SDK required

Yes

Yes

No

No

MCP native

Yes

No

No

No

Limitations

  • Prompt injection: Like all LLM-as-judge systems, adversarial inputs could attempt to manipulate reviewer verdicts. The anti-gaming validation layer mitigates superficial gaming, but determined adversarial inputs remain a challenge. For high-stakes use cases, combine with deterministic validation.

  • BYOK cost: Each review_output call makes 1 LLM API call; review_dual makes 3. Factor this into your pipeline costs.

Hosted API (Separate Product)

For teams that prefer HTTP integration, a hosted REST API with additional features (agent marketplace, context learning, workflows) is available at agentdesk.usedevtools.com.

Development

git clone https://github.com/Rih0z/agentdesk-mcp.git
cd agentdesk-mcp
npm install
npm test        # 35 tests
npm run build

License

MIT


Built by EZARK Consulting | Web Version

Available Tools

4 tools
execute_serviceC

Execute a service on the AgentDesk marketplace. Requires an AgentDesk API key for authentication. Pass service-specific input parameters.

Inputs: service_id (required), input (required), api_key.

list_servicesA

List all available services on the AgentDesk marketplace. Returns service catalog with pricing, quality scores, and capabilities. Filter by category, minimum quality score, maximum price, or capability.

Inputs: category, min_score, max_price, capability.

review_dualA

Dual adversarial review: two independent reviewers assess the output from different angles, then a merge agent combines their findings. Stricter than single review — if either reviewer finds a critical issue, the merged verdict is FAIL. Use for high-stakes outputs where quality is critical.

Inputs: output (required), criteria, review_type, model.

review_outputB

Adversarial quality review of any AI-generated output. An independent reviewer assumes the author made mistakes and actively looks for problems. Returns structured verdict (PASS/FAIL/CONDITIONAL_PASS), score (0-100), categorized issues with severity, and evidence-based checklist. Works for any output type: code, content, summaries, translations, data extraction, etc.

Inputs: output (required), criteria, review_type, model.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.6/5.0

    Scored across 4 tools

    Disambiguation5/5

    The four tools are clearly divided into two distinct categories: marketplace services (execute_service, list_services) and output review (review_dual, review_output). Even within the review category, the two tools have well-differentiated purposes (single vs. dual adversarial review), so there is no ambiguity.

    Naming Consistency5/5

    All tool names follow a consistent verb_noun pattern in snake_case: execute_service, list_services, review_dual, review_output. The naming is predictable and easy to understand.

    Tool Count4/5

    With 4 tools, the server covers two distinct functions. While each function could benefit from more tools (e.g., more marketplace operations or review management), the current count is reasonable for a focused server and does not feel excessive or insufficient.

    Completeness3/5

    The marketplace side only offers list and execute, lacking create, update, or delete operations for services. The review side provides two types of reviews but no ability to list or manage past reviews. These gaps limit the server's coverage for its implied domain.

    Maintenance

    ActivityNo data
    ResponsivenessSyncing

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      Not graded
      maintenance
      Enables AI-assisted code review with bias mitigation strategies through cross-model evaluation and bias-aware prompting. Detects AI-generated code from commit authors and provides structured reviews with security, performance, and maintainability analysis.
      -
    • A
      license
      A
      quality
      D
      maintenance
      AI-powered code review tool that detects AI-generated code defects invisible to traditional linters — hallucinated packages, deprecated APIs, cross-file contradictions, hidden security anti-patterns, and over-engineering. Works as a standalone CLI, GitHub Action, or MCP server. Supports TypeScript, Python, Java, Go, and Kotlin. Free for individuals, no API key required.
      4
      41
      Business Source 1.1
    • A
      license
      Not graded
      quality
      C
      maintenance
      Human-evaluation infrastructure for AI quality. 25,000+ blind human reviews by 200+ verified reviewers across 58 AI models — query the data via five MCP tools (get_model_scores, compare_models, get_flags, check_content, get_latest).
      2
      MIT
    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables AI-powered, zero-trust code review with multiple models, supporting single files, git diffs, and multiple files, with security, performance, and architecture checks across 10+ languages.
      15
      MIT