generate_skill_tests

Generate YAML test suites from SKILL.md files to validate AI agent skills across multiple test categories, then execute them for comprehensive evaluation.

Instructions

Auto-generate test cases from a SKILL.md file. Call this when the user asks to create tests for a skill — it reads the skill definition and generates a ready-to-run YAML test suite covering explicit, implicit, contextual, and negative test categories. After generating, call run_skill_test to execute them.

Input Schema

TableJSON Schema

Name	Required	Description
`skill_path`	Yes	Path to the SKILL.md file to generate tests from
`output_path`	No	Where to save the generated test YAML (default: same directory as SKILL.md)
`count`	No	Number of test cases to generate (default: 10)

Implementation Reference

evalview/mcp_server.py:438-447 (handler)

Implementation of the 'generate_skill_tests' tool which constructs the command line call to 'evalview skill generate-tests'.

elif name == "generate_skill_tests":
    skill_path = os.path.normpath(args.get("skill_path", ""))
    if not skill_path:
        return "Error: 'skill_path' is required."
    cmd = ["evalview", "skill", "generate-tests", skill_path, "--auto"]
    if args.get("output_path"):
        cmd += ["-o", os.path.normpath(args["output_path"])]
    if args.get("count"):
        cmd += ["-c", str(int(args["count"]))]

evalview/mcp_server.py:176-198 (schema)

Schema definition for 'generate_skill_tests' tool including input arguments.

{
    "name": "generate_skill_tests",
    "description": (
        "Auto-generate test cases from a SKILL.md file. "
        "Call this when the user asks to create tests for a skill — it reads the skill "
        "definition and generates a ready-to-run YAML test suite covering explicit, "
        "implicit, contextual, and negative test categories. "
        "After generating, call run_skill_test to execute them."
    ),
    "inputSchema": {
        "type": "object",
        "required": ["skill_path"],
        "properties": {
            "skill_path": {
                "type": "string",
                "description": "Path to the SKILL.md file to generate tests from",
            },
            "output_path": {
                "type": "string",
                "description": "Where to save the generated test YAML (default: same directory as SKILL.md)",
            },
            "count": {
                "type": "number",

EvalView