Skip to main content
Glama
Sign In
OrangePro MCP

OrangePro MCP

Official
by OrangeproAI
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

Generate grounded tests

orangepro_generate_tests

Generate up to five grounded, bucket-diverse tests from local evidence for targeted behaviors, returning code, run hints, and verifiable citations so agents can write and execute them.

Instructions

Generate a small capped set (1-5) of grounded tests from local evidence using your own model key (BYOK: OpenAI-compatible, Ollama, or Anthropic). Tests are diversified across lightweight LOCAL scenario buckets (happy_path, validation_error, edge_case, integration_flow, security_privacy, regression) chosen from the evidence — unjustified buckets are skipped, never padded. By default it focuses the top-gap behavior and returns up to limit bucket-diverse tests for it; pass multiple target_ids to split the budget across them. Each test reports its bucket plus VALIDATED grounding evidence (every cited entity resolved against the local graph with its kind, evidence strength, and source_ref) and discloses weak/candidate evidence; evidence_summary reports proof coverage and any broken citations, so the grounding is verifiable rather than asserted. If evidence is too thin, returns missing-evidence guidance instead of generic tests. Returns each test's code plus run_hints (a suggested file path + run command) and an agent_workflow: YOU (the agent) write each file and run it with your shell tools, then report pass/fail + stack traces — OrangePro never writes to or runs anything in the repo. Requires the repo's test framework/deps to already be installed.

Input Schema

NameRequiredDescriptionDefault
limitNoTotal tests to generate (1-5), spread across local scenario buckets. Default 3.
modelNoModel name override.
compareNoRun BOTH arms — prompt-only baseline vs Local KG, same model + system prompt — and return side-by-side scores (completeness, context awareness, accuracy, domain specificity) plus both test suites. Non-persisting testing view.
base_refNoPR-scoped generation: restrict to the behaviors the diff vs this ref touches (e.g. 'main') — for branch/PR review, target only the changed code. Requires a git repo + a real diff; returns structured guidance (status + guidance) instead of fabricating when there is none. Overrides target_ids.
providerNoBYOK provider override (auto-detected from env if omitted). Use 'deterministic' for an offline stand-in; without any provider configured, generation returns setup guidance instead of degrading.
frameworkNoFramework hint, e.g. 'playwright', 'vitest', 'pytest'.
workspaceNoLocal workspace root. Defaults to the current directory.
target_idsNoBehavior/requirement external ids to target. Default: the top-gap behavior. Pass several to split the budget across them (each gets a test when the budget allows, in priority order).
write_filesNoDeferred/premium: repo file writing is disabled in the first slice; always treated as false.
prompt_versionNoPrompt strategy. Defaults to v2; v5 is opt-in until corpus validation clears it as default.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • removedInput schema / additionalProperties
      Removed value: -false
  2. Addedv0.2.59

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false, destructiveHint=false, openWorldHint=true; the description adds the crucial contract that OrangePro never writes to or runs anything in the repo — the agent must write files and execute them itself. It also discloses the BYOK prerequisite, the no-provider fallback to setup guidance, and that evidence is validated against the local graph with broken citations surfaced. This is behavior an agent cannot infer from the schema or annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the verb, cap, and BYOK requirement before any detail, and each subsequent clause adds distinct information (buckets, evidence validation, thin-evidence fallback, agent workflow, prerequisites). It is a single dense paragraph rather than structured bullets, which makes the long evidence/run_hints passage harder to scan, but there is little true filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 10 parameters, no output schema, and openWorldHint=true, the description carries the burden well: it describes the returned fields (per-test bucket, validated evidence with kind/strength/source_ref, evidence_summary, code, run_hints, agent_workflow), the failure mode, and the environment prerequisite. Nothing essential for a correct first call appears to be missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description goes beyond it by explaining limit as a bucket-diverse budget, target_ids budget-splitting in priority order, and base_ref overriding target_ids. Provider auto-detection and the 'deterministic' offline stand-in are also given operational meaning. Only compare's full side-by-side scoring detail and framework/prompt_version nuances are left entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource plus scope: 'Generate a small capped set (1-5) of grounded tests from local evidence.' It further pins the mechanism (BYOK providers) and output character (bucket-diversified, evidence-grounded), so an agent knows exactly what it gets. It stops short of naming a sibling (e.g. find_test_gaps vs this generator) to route between them explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete operating guidance: default targets the top-gap behavior, pass multiple target_ids to split budget, use base_ref for PR/branch-scoped generation (and that it overrides target_ids), use compare for a prompt-only vs KG baseline. It also states the thin-evidence path returns guidance rather than generic tests. It never states when NOT to use this tool or names the sibling to pick instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools