Skip to main content
Glama
kirti676

API Tester MCP

by kirti676

API Tester MCP Server

npm (scoped) npm downloads License: MIT

A comprehensive Model Context Protocol (MCP) server for QA/SDET engineers that provides API testing capabilities with Swagger/OpenAPI and Postman collection support.

๐ŸŽ‰ Now available on NPM! Install with npx @kirti676/api-tester-mcp@latest

๐Ÿ†• What's New

  • โœ… Enhanced Progress Tracking - Real-time progress with completion percentages and ETA

  • โœ… Visual Progress Bars - ASCII progress bars with milestone notifications

  • โœ… Performance Metrics - Throughput calculations and execution summaries

  • โœ… Published on NPM - Install instantly with NPX

  • โœ… VS Code Integration - One-click installation buttons

  • โœ… Simplified Setup - No manual Python installation required

  • โœ… Cross-Platform - Works on Windows, macOS, and Linux

  • โœ… Auto-Updates - Always get the latest version with @latest

Related MCP server: Codebase Insights MCP Server

๐Ÿš€ Getting Started

๐Ÿ“ฆ Installation

The API Tester MCP server can be used directly with npx without any installation:

npx @kirti676/api-tester-mcp@latest

โšก Quick Install:

Install in VS Code Install in VS Code Insiders

๐Ÿค– Claude Desktop

Follow the MCP install guide, use the standard config below:

{
  "mcpServers": {
    "api-tester": {
      "command": "npx",
      "args": ["@kirti676/api-tester-mcp@latest"]
    }
  }
}

๐Ÿ”— Other MCP Clients

The standard configuration works with most MCP clients:

{
  "mcpServers": {
    "api-tester": {
      "command": "npx",
      "args": ["@kirti676/api-tester-mcp@latest"]
    }
  }
}

๐Ÿ–ฅ๏ธ Supported Clients:

๐Ÿ Python Installation (Alternative)

pip install api-tester-mcp

๐Ÿ’ป From Source

git clone https://github.com/kirti676/api_tester_mcp.git
cd api_tester_mcp
npm install

โšก Quick Start

Try the API Tester MCP server immediately:

# Run the server
npx @kirti676/api-tester-mcp@latest

# Check version
npx @kirti676/api-tester-mcp@latest --version

# Get help
npx @kirti676/api-tester-mcp@latest --help

For MCP clients like Claude Desktop, use this configuration:

{
  "mcpServers": {
    "api-tester": {
      "command": "npx",
      "args": ["@kirti676/api-tester-mcp@latest"]
    }
  }
}

โœจ Features

  • ๐Ÿ“ฅ Input Support: OpenAPI/Swagger documents, Postman collections, and GraphQL schemas

  • ๐Ÿ”„ Test Generation: Automatic API and Load test scenario generation

  • ๐ŸŒ Multi-Language Support: Generate tests in TypeScript/Playwright, JavaScript/Jest, Python/pytest, and more

  • โšก Test Execution: Run generated tests with detailed reporting

  • ๐Ÿ” Smart Auth Detection: Automatic environment variable analysis and setup guidance

  • ๐Ÿ” Authentication: Bearer token and API key support via set_env_vars

  • ๐Ÿ“Š HTML Reports: Beautiful, accessible reports via MCP resources

  • ๐Ÿ“ˆ Real-time Progress: Live updates with progress bars and completion percentages

  • โฑ๏ธ ETA Calculations: Estimated time to completion for all operations

  • ๐ŸŽฏ Milestone Tracking: Special notifications at key progress milestones (25%, 50%, 75%, etc.)

  • ๐Ÿ“Š Performance Metrics: Throughput calculations and execution summaries

  • โœ… Schema Validation: Request body generation from schema examples

  • ๐ŸŽฏ Assertions: Per-endpoint status code assertions (2xx, 4xx, 5xx)

  • ๐Ÿ“ฆ Project Generation: Complete project scaffolding with dependencies and configuration

๐ŸŒ Multi-Language Test Generation

The API Tester MCP now supports generating test code in multiple programming languages and testing frameworks:

๐Ÿ”ง Supported Language/Framework Combinations

Language

Framework

Description

Use Case

๐Ÿ“˜ TypeScript

๐ŸŽญ Playwright

Modern E2E testing with excellent API support

๐Ÿข Enterprise web applications

๐Ÿ“˜ TypeScript

๐Ÿš€ Supertest

Express.js focused API testing

๐ŸŸข Node.js backend services

๐Ÿ“™ JavaScript

๐Ÿƒ Jest

Popular testing framework with good ecosystem

๐Ÿ”ง General API testing

๐Ÿ“™ JavaScript

๐ŸŒฒ Cypress

E2E testing with great developer experience

๐ŸŒ Full-stack applications

๐Ÿ Python

๐Ÿงช pytest

Comprehensive testing with fixtures & plugins

๐Ÿ“Š Data-heavy APIs & ML services

๐Ÿ Python

๐Ÿ“ก requests

Simple HTTP testing for quick validation

โšก Rapid prototyping & scripts

๐ŸŽฏ Language Selection Workflow

// 1. Get available languages and frameworks
const languages = await mcp.call("get_supported_languages");

// 2. Choose your preferred combination
await mcp.call("ingest_spec", {
  spec_type: "openapi",
  file_path: "./path/to/your/api-spec.json",
  preferred_language: "typescript",    // python, typescript, javascript
  preferred_framework: "playwright"     // varies by language
});

// 3. Generate test cases with code
await mcp.call("generate_test_cases", {
  language: "typescript",
  framework: "playwright"
});

// 4. Get complete project setup
await mcp.call("generate_project_files", {
  language: "typescript",
  framework: "playwright",
  project_name: "my-api-tests",
  include_examples: true
});

๐Ÿ“ Generated Project Structure

The generate_project_files tool creates a complete, ready-to-run project:

๐Ÿ“˜ TypeScript + Playwright:

my-api-tests/
โ”œโ”€โ”€ ๐Ÿ“ฆ package.json          # Dependencies & scripts
โ”œโ”€โ”€ โš™๏ธ playwright.config.ts  # Playwright configuration
โ”œโ”€โ”€ ๐Ÿ“‚ tests/
โ”‚   โ””โ”€โ”€ ๐Ÿงช api.spec.ts      # Generated test code
โ””โ”€โ”€ ๐Ÿ“– README.md            # Setup instructions

๐Ÿ Python + pytest:

my-api-tests/
โ”œโ”€โ”€ ๐Ÿ“‹ requirements.txt     # Python dependencies
โ”œโ”€โ”€ โš™๏ธ pytest.ini         # pytest configuration
โ”œโ”€โ”€ ๐Ÿ“‚ tests/
โ”‚   โ””โ”€โ”€ ๐Ÿงช test_api.py    # Generated test code
โ””โ”€โ”€ ๐Ÿ“– README.md          # Setup instructions

๐Ÿ“™ JavaScript + Jest:

my-api-tests/
โ”œโ”€โ”€ ๐Ÿ“ฆ package.json       # Dependencies & scripts
โ”œโ”€โ”€ โš™๏ธ jest.config.js     # Jest configuration
โ”œโ”€โ”€ ๐Ÿ“‚ tests/
โ”‚   โ””โ”€โ”€ ๐Ÿงช api.test.js   # Generated test code
โ””โ”€โ”€ ๐Ÿ“– README.md         # Setup instructions

๐ŸŽฏ Framework-Specific Features

  • ๐ŸŽญ Playwright: Browser automation, parallel execution, detailed reporting

  • ๐Ÿƒ Jest: Snapshot testing, mocking, watch mode for development

  • ๐Ÿงช pytest: Fixtures, parametrized tests, extensive plugin ecosystem

  • ๐ŸŒฒ Cypress: Interactive debugging, time-travel debugging, real browser testing

  • ๐Ÿš€ Supertest: Express.js integration, middleware testing

  • ๐Ÿ“ก requests: Simple API calls, session management, authentication helpers

๐Ÿ“ˆ Progress Tracking

The API Tester MCP includes comprehensive progress tracking for all operations:

๐Ÿ“Š Visual Progress Indicators

๐ŸŽฏ API Test Execution: [โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘] 50.0% (5/10) | ETA: 2.5s - GET /api/users โœ…

๐Ÿ”ฅ Features:

  • ๐Ÿ“Š Progress Bars: ASCII progress bars with filled/empty indicators

  • ๐Ÿ“ˆ Completion Percentages: Real-time percentage completion

  • โฐ ETA Calculations: Estimated time to completion based on current performance

  • ๐ŸŽฏ Milestone Notifications: Special highlighting at key progress points

  • โšก Performance Metrics: Throughput and timing statistics

  • ๐Ÿ“‹ Operation Context: Detailed information about current step being executed

โœ… Available for:

  • ๐ŸŽฌ Scenario generation

  • ๐Ÿงช Test case generation

  • ๐Ÿš€ API test execution

  • โšก Load test execution

  • ๐Ÿ”„ All long-running operations

๐Ÿ› ๏ธ MCP Tools

The server provides 11 comprehensive MCP tools with detailed parameter specifications:

1. ๐Ÿ“ฅ ingest_spec - Load API Specifications

Load OpenAPI/Swagger, Postman collections, or GraphQL schemas with language/framework preferences

{
  "spec_type": "openapi",           // openapi, swagger, postman, graphql (optional, auto-detected)
  "file_path": "./api-spec.json",   // Path to JSON, YAML, or GraphQL schema file (required)
  "preferred_language": "python",   // python, typescript, javascript (optional, default: python)
  "preferred_framework": "requests" // pytest, requests, playwright, jest, cypress, supertest (optional, default: requests)
}

2. ๐Ÿ”ง set_env_vars - Configure Authentication & Environment

Set environment variables with automatic validation and guidance

{
  "variables": {},                  // Dictionary of custom environment variables (optional)
  "baseUrl": null,                 // API base URL (optional)
  "auth_bearer": null,             // Bearer/JWT token (optional)
  "auth_apikey": null,             // API key (optional)
  "auth_basic": null,              // Base64 encoded credentials (optional)
  "auth_username": null,           // Username for basic auth (optional)
  "auth_password": null            // Password for basic auth (optional)
}

3. ๐ŸŽฌ generate_scenarios - Create Test Scenarios

Generate test scenarios from ingested specifications

{
  "include_negative_tests": true,   // Generate failure scenarios (default: true)
  "include_edge_cases": true        // Generate boundary conditions (default: true)
}

4. ๐Ÿงช generate_test_cases - Convert to Executable Tests

Convert scenarios to executable test cases in preferred language/framework

{
  "scenario_ids": null              // Array of scenario IDs or null for all (optional)
}

5. ๐Ÿš€ run_api_tests - Execute API Tests

Execute API tests with detailed results and reporting

{
  "test_case_ids": null,           // Array of test case IDs or null for all (optional)
  "max_concurrent": 10             // Number of concurrent requests 1-50 (default: 10)
}

6. โšก run_load_tests - Execute Performance Tests

Execute load/performance tests with configurable parameters

{
  "test_case_ids": null,           // Array of test case IDs or null for all (optional)
  "duration": 60,                  // Test duration in seconds (default: 60)
  "users": 10,                     // Number of concurrent virtual users (default: 10)
  "ramp_up": 10                    // Ramp up time in seconds (default: 10)
}

7. ๐ŸŒ get_supported_languages - List Language/Framework Options

Get list of supported programming languages and testing frameworks

// No parameters required
{}

8. ๐Ÿ“ฆ generate_project_files - Generate Complete Projects

Generate complete project structure with dependencies and configuration

{
  "project_name": null,            // Project folder name (optional, auto-generated if null)
  "include_examples": true         // Include example test files (default: true)
}

9. ๐Ÿ“ get_workspace_info - Workspace Information

Get information about workspace directory and file generation locations

// No parameters required
{}

10. ๐Ÿ” debug_file_system - File System Diagnostics

Get comprehensive workspace information and file system diagnostics

// No parameters required
{}

11. ๐Ÿ“Š get_session_status - Session Status & Progress

Retrieve current session information with progress details

// No parameters required
{}

๐Ÿ“š MCP Resources

  • file://reports - List all available test reports

  • file://reports/{report_id} - Access individual HTML test reports

๐Ÿ’ก MCP Prompts

  • create_api_test_plan - Generate comprehensive API test plans

  • analyze_test_failures - Analyze test failures and provide recommendations

๐Ÿ” Smart Environment Variable Analysis

The API Tester MCP now automatically analyzes your API specifications to detect required environment variables and provides helpful setup guidance:

๐ŸŽฏ Automatic Detection

  • ๐Ÿ” Authentication Schemes: Bearer tokens, API keys, Basic auth, OAuth2

  • ๐ŸŒ Base URLs: Extracted from specification servers/hosts

  • ๐Ÿ”— Template Variables: Postman collection variables like {{baseUrl}}, {{authToken}}

  • ๐Ÿ“ Path Parameters: Dynamic values in paths like /users/{userId}

๐Ÿ’ก Smart Suggestions

// 1. Ingest specification - automatic analysis included
const result = await mcp.call("ingest_spec", {
  spec_type: "openapi",
  file_path: "./api-specification.json"
});

// Check the setup message for immediate guidance
console.log(result.setup_message);
// "โš ๏ธ 2 required environment variable(s) detected..."

// 2. Get detailed setup instructions
const suggestions = await mcp.call("get_env_var_suggestions");
console.log(suggestions.setup_instructions);
// Provides copy-paste ready configuration examples

๐ŸŽฏ Default Parameter Keys

All MCP tools now provide helpful default parameter keys to guide users on what values they can set:

๐Ÿ”ง Environment Variables (set_env_vars)

๐Ÿ”‘ ALL PARAMETERS ARE OPTIONAL - Provide only what you need:

// Option 1: Just the base URL
await mcp.call("set_env_vars", {
  baseUrl: "https://api.example.com/v1"
});

// Option 2: Just authentication
await mcp.call("set_env_vars", {
  auth_bearer: "your-jwt-token-here"
});

// Option 3: Multiple parameters
await mcp.call("set_env_vars", {
  baseUrl: "https://api.example.com/v1",
  auth_bearer: "your-jwt-token",
  auth_apikey: "your-api-key"
});

// Option 4: Using variables dict for custom values
await mcp.call("set_env_vars", {
  variables: {
    "baseUrl": "https://api.example.com/v1",
    "custom_header": "custom-value"
  }
});

๐ŸŒ Language & Framework Selection

Default values help you understand available options:

// Ingest with defaults shown
await mcp.call("ingest_spec", {
  spec_type: "openapi",        // openapi, swagger, postman
  file_path: "./api-spec.json", // Path to JSON or YAML specification file
  preferred_language: "python", // python, typescript, javascript
  preferred_framework: "requests" // pytest, requests, playwright, jest, cypress, supertest
});

// Project generation with defaults
await mcp.call("generate_project_files", {
  language: "python",          // python, typescript, javascript
  framework: "requests",       // Framework matching the language
  project_name: "api-tests",   // Project folder name
  include_examples: true       // Include example test files
});

โšก Test Execution Parameters

Clear defaults for performance tuning:

// API tests with concurrency control
await mcp.call("run_api_tests", {
  test_case_ids: null,        // ["test_1", "test_2"] or null for all
  max_concurrent: 10          // Number of concurrent requests (1-50)
});

// Load tests with performance parameters  
await mcp.call("run_load_tests", {
  test_case_ids: null,        // ["test_1", "test_2"] or null for all
  duration: 60,               // Test duration in seconds
  users: 10,                  // Number of concurrent virtual users
  ramp_up: 10                 // Ramp up time in seconds
});

๐Ÿ”ง Configuration Example

// NEW: Check supported languages and frameworks
const languages = await mcp.call("get_supported_languages");
console.log(languages.supported_combinations);

// Ingest specification with language preferences
await mcp.call("ingest_spec", {
  spec_type: "openapi",
  file_path: "./openapi-specification.json",
  preferred_language: "typescript",
  preferred_framework: "playwright"
});

// Set environment variables for authentication
await mcp.call("set_env_vars", {
  variables: {
    "baseUrl": "https://api.example.com",
    "auth_bearer": "your-bearer-token",
    "auth_apikey": "your-api-key"
  }
});

// Generate test scenarios
await mcp.call("generate_scenarios", {
  include_negative_tests: true,
  include_edge_cases: true
});

// Generate test cases in TypeScript/Playwright
await mcp.call("generate_test_cases", {
  language: "typescript",
  framework: "playwright"
});

// Generate complete project files
await mcp.call("generate_project_files", {
  language: "typescript",
  framework: "playwright",
  project_name: "my-api-tests",
  include_examples: true
});

// Run API tests (still works with existing execution engine)
await mcp.call("run_api_tests", {
  max_concurrent: 5
});

๐Ÿš€ Complete Workflow Example

Here's a complete example of testing the Petstore API:

# 1. Start the MCP server
npx @kirti676/api-tester-mcp@latest

Then in your MCP client (like Claude Desktop):

// 1. Load the Petstore OpenAPI spec
await mcp.call("ingest_spec", {
  spec_type: "openapi",
  file_path: "./examples/petstore_openapi.json"
});

// 2. Set environment variables
await mcp.call("set_env_vars", {
  pairs: {
    "baseUrl": "https://petstore.swagger.io/v2",
    "auth_apikey": "special-key"
  }
});

// 3. Generate test cases
const tests = await mcp.call("get_generated_tests");

// 4. Run API tests
const result = await mcp.call("run_api_tests");

// 5. View results in HTML report
const reports = await mcp.call("list_resources", {
  uri: "file://reports"
});

๐Ÿ“– Usage Examples

๐Ÿ”„ Basic API Testing Workflow

  1. ๐Ÿ“ฅ Ingest API Specification

    {
      "tool": "ingest_spec",
      "params": {
        "spec_type": "openapi",
        "content": "{ ... your OpenAPI spec ... }"
      }
    }
  2. ๐Ÿ” Configure Authentication

    {
      "tool": "set_env_vars", 
      "params": {
        "variables": {
          "auth_bearer": "your-token",
          "baseUrl": "https://api.example.com"
        }
      }
    }
  3. ๐Ÿš€ Generate and Run Tests

    {
      "tool": "generate_scenarios",
      "params": {
        "include_negative_tests": true
      }
    }
  4. ๐Ÿ“Š View Results

    • ๐Ÿ“„ Access HTML reports via MCP resources

    • ๐Ÿ“ˆ Get session status and statistics

๐Ÿš€ GraphQL API Testing Workflow

  1. ๐Ÿ“ฅ Ingest GraphQL Schema

    {
      "tool": "ingest_spec",
      "params": {
        "spec_type": "graphql",
        "file_path": "./schema.graphql"
      }
    }
  2. ๐Ÿ” Configure GraphQL Endpoint

    {
      "tool": "set_env_vars", 
      "params": {
        "graphqlEndpoint": "https://api.example.com/graphql",
        "auth_bearer": "your-jwt-token"
      }
    }
  3. ๐Ÿงช Generate GraphQL Tests

    {
      "tool": "generate_test_cases",
      "params": {
        "preferred_language": "python",
        "preferred_framework": "pytest"
      }
    }
  4. ๐Ÿ“Š Execute GraphQL Tests

    {
      "tool": "run_api_tests",
      "params": {
        "max_concurrent": 5
      }
    }

โšก Load Testing

{
  "tool": "run_load_tests",
  "params": {
    "users": 10,
    "duration": 60,
    "ramp_up": 10
  }
}

๐Ÿ” Test Generation Features

  • โœ… Positive Tests: Valid requests with expected 2xx responses

  • โŒ Negative Tests: Invalid authentication (401), wrong methods (405)

  • ๐ŸŽฏ Edge Cases: Large payloads, boundary conditions

  • ๐Ÿ—๏ธ Schema-based Bodies: Automatic request body generation from OpenAPI schemas

  • ๐Ÿ” Comprehensive Assertions: Status codes, response times, content validation

๐Ÿ“Š HTML Reports

Generated reports include:

  • ๐Ÿ“ˆ Test execution summary with pass/fail statistics

  • โฑ๏ธ Detailed test results with timing information

  • ๐Ÿ” Assertion breakdowns and error details

  • ๐Ÿ‘๏ธ Response previews and debugging information

  • ๐Ÿ“ฑ Mobile-friendly responsive design

๐Ÿ”’ Authentication Support

  • ๐ŸŽซ Bearer Tokens: auth_bearer environment variable

  • ๐Ÿ”‘ API Keys: auth_apikey environment variable (sent as X-API-Key header)

  • ๐Ÿ‘ค Basic Auth: auth_basic environment variable

๐Ÿ”ง Requirements

  • ๐Ÿ Python: 3.8 or higher

  • ๐ŸŸข Node.js: 14 or higher (for npm installation)

๐Ÿ“ฆ Dependencies

๐Ÿ Python Dependencies

  • ๐Ÿš€ fastmcp>=0.2.0

  • ๐Ÿ“Š pydantic>=2.0.0

  • ๐ŸŒ requests>=2.28.0

  • โœ… jsonschema>=4.0.0

  • ๐Ÿ“ pyyaml>=6.0

  • ๐ŸŽจ jinja2>=3.1.0

  • โšก aiohttp>=3.8.0

  • ๐ŸŽญ faker>=19.0.0

๐ŸŸข Node.js Dependencies

  • โœจ None (self-contained package)

๐Ÿ”ง Troubleshooting

โ— Common Issues

๐Ÿ“ฆ NPX Command Not Working

# If npx command fails, try:
npm install -g @kirti676/api-tester-mcp@latest

# Or run directly:
node ./node_modules/@kirti676/api-tester-mcp/cli.js

๐Ÿ Python Not Found

# Make sure Python 3.8+ is installed and in PATH
python --version

# Install Python dependencies manually if needed:
pip install fastmcp>=0.2.0 pydantic>=2.0.0 requests>=2.28.0

๐Ÿ”— MCP Client Connection Issues

  • โœ… Ensure the MCP server is running on stdio transport (default)

  • ๐Ÿ”„ Check that your MCP client supports the latest MCP protocol version

  • ๐Ÿ“ Verify the configuration JSON syntax is correct

๐Ÿ†˜ Getting Help

  1. ๐Ÿ“– Check the Examples directory for working configurations

  2. ๐Ÿ” Run with --verbose flag for detailed logging

  3. ๐Ÿ› Report issues on GitHub Issues

๐Ÿค Contributing

  1. Fork the repository

  2. Create your feature branch (git checkout -b feature/amazing-feature)

  3. Commit your changes (git commit -m 'Add some amazing feature')

  4. Push to the branch (git push origin feature/amazing-feature)

  5. Open a Pull Request

๐Ÿ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

๐Ÿ› Issues & Support

๐Ÿ“ˆ Roadmap

  • Multi-Language Test Generation - TypeScript/Playwright, JavaScript/Jest, Python/pytest support โœจ NEW!

  • Complete Project Generation - Full project scaffolding with dependencies and configuration โœจ NEW!

  • GraphQL API support - Supports GraphQL Schemas โœจ NEW!

  • Additional authentication methods (OAuth2, JWT)

  • Go/Golang test generation (with testify/ginkgo)

  • C#/.NET test generation (with NUnit/xUnit)

  • Performance monitoring and alerting

  • Integration with CI/CD pipelines (GitHub Actions, Jenkins)

  • Advanced test data generation from examples and schemas

  • API contract testing with Pact support

  • Mock server generation for development

ยฉ 2025 kirti676. All rights reserved.

This repository and its contents are protected by copyright law. For permission to reuse, reference, or redistribute any part of this project, please contact the owner at kirti676@outlook.com.

โœ… Allowed without permission:

  • Personal learning and experimentation

  • Contributing back to this repository via Pull Requests

โ“ Requires permission:

  • Commercial use or integration

  • Redistribution in modified form

  • Publishing derived works

For licensing inquiries, collaboration opportunities, or permission requests, reach out to kirti676@outlook.com.


โญ Star this repo ๐Ÿด Fork this repo

๐Ÿš€ Built with โค๏ธ for QA/SDET engineers worldwide ๐ŸŒ

Available Tools

11 tools
debug_file_systemA

Get comprehensive workspace information and file system diagnostics. Shows where files will be saved without creating directories.

Returns: Dictionary with workspace information and file system diagnostic details

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden. It reveals non-destructive behavior ('without creating directories') and indicates it's a read-only diagnostic. However, lacks details on response structure or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences, front-loaded with key purpose. Slightly vague on return details but overall concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no parameters and output schema present, the description covers the main purpose and behavior. Could add more on diagnostics scope but sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, schema coverage 100%. Description adds meaning by explaining the tool shows save locations without creating directories, going beyond the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool gets comprehensive workspace info and file system diagnostics, including showing where files will be saved without creating directories. This distinguishes it from siblings like get_workspace_info.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this vs. alternatives. The description implies debugging file system behavior but doesn't mention when not to use it or refer to siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_project_filesB

Generate complete project files using the language and framework specified in ingest_spec. Automatically reuses test cases and parameters from the current session if available.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, and the description fails to disclose behavioral traits such as whether files are overwritten, permissions needed, or side effects of reusing session data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently written in two sentences with no redundancy, clearly stating the action and a notable behavior.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

While the output schema covers return values, the description omits preconditions (e.g., need for a valid ingest_spec) and does not address potential destructive actions, making it adequate but not thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds no explanation for the two parameters (project_name, include_examples) beyond the input schema. With 0% schema description coverage, this is a critical gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool generates complete project files based on ingest_spec, distinguishing it from sibling tools like generate_scenarios and generate_test_cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when an ingest_spec is present and session data is available, but does not explicitly state when to avoid using it or provide alternatives among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_scenariosB

Generate test scenarios from the ingested API specification.

NOTE: This function automatically saves the generated scenarios to both the output directory and the current workspace for easy access.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the automatic saving behavior to output directory and workspace, which is useful. However, it does not mention any potential side effects, required permissions, or whether the operation is destructive. With no annotations, this level of transparency is adequate but not comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exceptionally concise with two sentences. The first sentence clearly states the primary action, and the second adds a critical behavioral note. No unnecessary words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (nested params, boolean options) and the presence of an output schema (not shown), the description is adequate but incomplete. It lacks parameter details and does not describe the output format. For a tool with siblings like 'generate_test_cases', more context would be beneficial.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage, and the description does not explain the two boolean parameters ('include_negative_tests', 'include_edge_cases') or their defaults. While the parameter names are somewhat self-explanatory, the description should clarify their effect on scenario generation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: generating test scenarios from an ingested API specification. It effectively distinguishes from sibling tools like 'generate_test_cases' and 'generate_project_files' by specifying the input (API spec) and output (scenarios).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'generate_test_cases'. The description lacks any 'when to use' or 'when not to use' information, leaving the AI agent to infer context from sibling tool names alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generate_test_casesA

Generate executable test cases from scenarios using the language and framework specified in ingest_spec.

NOTE: This function automatically saves the generated test cases to both the output directory and the current workspace for easy access.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description discloses the important side effect of auto-saving to both output directory and current workspace. However, it does not mention behavior when scenario_ids is null, whether files are overwritten, or error handling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with the primary purpose, followed by a critical behavioral note. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained. But the description omits parameter details and does not address the context of how scenario_ids relate to scenarios from generate_scenarios, leaving gaps for a single-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must explain parameters. It mentions 'from scenarios' but does not clarify the meaning of scenario_ids, its format, or that it is optional. The agent lacks sufficient parameter guidance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'generate', the resource 'executable test cases from scenarios', and the dependency on 'ingest_spec' for language and framework. This distinguishes it from sibling tools like generate_scenarios, which produce scenarios rather than tests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies that ingest_spec should be called first to set language/framework, but does not explicitly state prerequisites or alternatives like generate_scenarios for creating scenarios. No guidance on when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_session_statusB

Get current session status and information with progress details.

Returns: Dictionary with current session information including progress

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden of disclosing behavioral traits. It only mentions return format (dictionary with progress details) but does not indicate safety (read vs write), idempotency, or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no extraneous words. First sentence states the core purpose, second specifies return format. Every sentence is needed and efficiently worded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity (no parameters, output schema exists), the description is sufficient to convey what the tool does and returns. However, it could mention any prerequisites or side effects for completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the input schema is fully covered (empty). The description adds no parameter information but correctly implies no inputs are needed. For zero-parameter tools, baseline is 4.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves 'current session status' with 'progress details', using the verb 'Get' and a specific resource. It distinguishes from siblings like get_workspace_info by focusing on session context, though the term 'session' could be more defined.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives, nor are any conditions for use or non-use stated. The description only states what it does, leaving the agent to infer context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_supported_languagesB

Get list of supported programming languages and testing frameworks.

Returns: Dictionary with supported language/framework combinations and their descriptions

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden for behavioral disclosure. It only states the return type (dictionary) and content, but does not mention side effects, required permissions, rate limits, or whether the tool is read-only. However, the simple retrieval nature suggests low risk.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very concise: two sentences with no extraneous information. The first sentence front-loads the purpose, and the second specifies the return format. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (no parameters, output schema present), the description is nearly complete. It specifies the return type and content. However, it could mention that the list is for test generation purposes, but this is minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters (0 params, 100% schema coverage), so baseline is 4. The description adds no parameter info, which is acceptable as no parameters exist. It does not detract from the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves a list of supported programming languages and testing frameworks. It is specific and distinguishes from sibling tools like run_api_tests or generate_test_cases, which involve execution rather than listing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. For instance, it does not mention that this tool should be called before generating test cases to check language support. The description lacks explicit when-to-use or when-not-to-use instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_workspace_infoA

Get information about the current workspace directory and file generation locations.

Returns: Dictionary with workspace information including current directory and whether it was set from an ingested API specification file.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It explains the return format (dictionary with current directory and ingest flag) but does not disclose any other behavioral traits such as side effects, permissions, or state dependencies. The description adds basic context but lacks depth for full transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences that front-load the purpose and then detail the return format. Every sentence is necessary and there is zero waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no parameters and an output schema, the description is fairly complete. It covers the key return fields. However, it omits any potential error conditions or prerequisites. Despite this, for a simple info-retrieval tool, it provides sufficient context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With zero parameters, the baseline is 4. The description adds significant value by explaining the return structure beyond the trivial input schema. It specifies that the output includes the current directory and whether it was set from an ingested API specification file, which is meaningful for the agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Get information') and resource ('current workspace directory and file generation locations'). It directly addresses what the tool does and distinguishes it from siblings like 'debug_file_system' or 'generate_*' which have different scopes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention exclusion criteria, prerequisites, or context where other tools might be more appropriate. Given the lack of usage recommendations, the score is low.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

ingest_specA

Ingest an API specification (OpenAPI/Swagger or Postman collection) from a file. Automatically analyzes the specification and suggests required environment variables.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so the description is the sole source of behavioral info. It mentions automatic analysis and suggestion of env variables, which is useful. However, it does not disclose potential failures (e.g., invalid file format), side effects, or whether the spec is stored permanently. The description is adequate but not thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the main action and resource, the second adds a key behavior. No unnecessary words, front-loads the core purpose. Ideal length for a tool description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite having an output schema, the description omits important context such as file path requirements, supported formats beyond the name drop, and the relationship to other tools (e.g., ingesting before running tests). For a complex operation like file ingestion, more context is needed to avoid misuse.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It mentions 'OpenAPI/Swagger or Postman collection' which relates to spec_type, but does not clarify the other three parameters (file_path, preferred_language, preferred_framework) or their defaults. The description adds some context but insufficiently maps to the schema, especially for a tool with 4 hidden parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Ingest' and the resource 'API specification (OpenAPI/Swagger or Postman collection)'. It also specifies the source 'from a file' and an additional behavior 'automatically analyzes and suggests required environment variables'. This distinguishes it from siblings like run_api_tests or set_env_vars, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. The description implies it is for ingesting API specs, but does not specify prerequisites (e.g., file existence) or order of operations relative to sibling tools like run_api_tests or set_env_vars. The context is clear but incomplete for an AI agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_api_testsC

Execute API tests and generate results.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description should fully disclose behavioral traits. It does not mention side effects, return format, auth needs, or whether results are stored or streamed. The output schema exists but is unaddressed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise (one sentence), but at the cost of omitting crucial details. It is not front-loaded with key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema and multiple sibling tools, the description does not provide enough context. The agent does not know what 'results' look like or how this tool relates to 'run_load_tests'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 0% property description coverage, and the description adds no meaning to 'test_case_ids' or 'max_concurrent'. The agent cannot infer what these parameters do beyond the schema types and defaults.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it executes API tests and generates results. However, it does not explicitly distinguish from the sibling 'run_load_tests', relying on the word 'API' vs 'load' which is subtle.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like 'run_load_tests'. No prerequisites, context, or when-not-to-use information is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_load_testsC

Execute load tests with specified parameters.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only states 'execute load tests', omitting side effects, output, permissions, or any constraints. The existence of an output schema is not mentioned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence, which is concise but lacks critical detail. It earns its place but does not leverage the format to provide structured, front-loaded information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the nested input schema, lack of annotations, and an output schema (unreferenced), the description is severely incomplete. It fails to explain parameter roles, expected behavior, or return value, making it inadequate for correct tool invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description does not elaborate on any of the four parameters (test_case_ids, duration, users, ramp_up). Schema coverage is 0%, so the agent gets no additional meaning beyond the raw schema, which includes defaults but no explanations.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Execute load tests with specified parameters' clearly states the verb (execute) and resource (load tests), but does not differentiate from sibling tools like 'run_api_tests' or 'debug_file_system'. It is specific enough to understand the general action.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives, or any prerequisites or exclusions. The agent must infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

set_env_varsB

Set environment variables for authentication and configuration.

This function automatically analyzes the API specification to provide proper validation and context about required/suggested variables.

ParametersJSON Schema
NameRequiredDescriptionDefault
paramsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description must carry full burden. Mentions analyzing API spec but omits critical behavioral details like persistence, override behavior, security implications, or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the main verb. Efficient, though the second sentence could be more specific.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (nested object, many parameters, no annotations), the description is too brief. Missing output schema details and fails to explain how parameters interact or what the tool actually does beyond setting variables.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 0% description coverage for parameters. Description only mentions 'authentication and configuration' at a high level, failing to clarify the role of each parameter (variables, baseUrl, auth_*).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Cleary states it sets environment variables for authentication and configuration, with a specific verb and resource. Distinguishes from sibling tools which are about debugging, testing, and file management.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies use for setting up environment variables with API spec context, but lacks explicit when-to-use/when-not-to-use guidance or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 11 tool updatesv1.5.3
    • First observeddebug_file_system
    • First observedgenerate_project_files
    • First observedgenerate_scenarios
    • First observedgenerate_test_cases
    • First observedget_session_status
    • First observedget_supported_languages
    • First observedget_workspace_info
    • First observedingest_spec
    • First observedrun_api_tests
    • First observedrun_load_tests
    • First observedset_env_vars

TDQS

B3.4/5.0
Disambiguation4/5

Most tools have distinct purposes, but debug_file_system and get_workspace_info both provide workspace information, which could cause minor confusion. The descriptions help differentiate them.

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in snake_case (e.g., generate_scenarios, run_api_tests). No mixed conventions or ambiguous verbs.

Tool Count5/5

11 tools is appropriate for an API testing server. Each tool serves a clear purpose in the workflow without being excessive or insufficient.

Completeness4/5

The tool set covers the core API testing workflow: ingestion, setup, scenario/test generation, execution, and diagnostics. Minor gaps include inability to view generated content directly or delete resources, but these are not critical.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    C
    maintenance
    Analyzes API codebases from GitHub and Bitbucket repositories to generate Postman collections, business reports, and detailed code insights. Supports multiple frameworks including FastAPI, Spring Boot, Flask, Express, and OpenAPI/Swagger specifications.
    MIT
  • F
    license
    Not graded
    quality
    C
    maintenance
    A full-stack API automation testing server that parses OpenAPI/Swagger/Postman/HAR specs, generates comprehensive test scenarios and executable code, and provides AI-powered review and auto-fix.
    -
  • F
    license
    Not graded
    quality
    B
    maintenance
    Enables automated QA testing by running a pipeline of AI agents that generate test scenarios, architect test layers, write Playwright tests, and review code, all grounded in feature requirements and API contracts.
    -

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/kirti676/api_tester_mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server