service-test-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@service-test-mcpcall GetUser with id 42 and assert the SOAP response status is 200"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
service-test-mcp-simple (service-test-mcp)
WCF/SOAP ve REST servislerini test etmek için MCP sunucusu. VS Code'da GitHub Copilot (Agent modu, custom agent'lar) ile ya da VS Code'dan bağımsız herhangi bir MCP istemcisi / AI ajanıyla kullanılır. sql-mcp-simple ve mongo-mcp-simple ile aynı yaklaşım: ajan servis çağrılarını PowerShell / Invoke-WebRequest / curl çalıştırmadan, doğrudan bu sunucunun araçlarıyla yapar.
SOAP / WCF | WSDL keşfi, operasyon şeması ve örnek parametre, çağrı, SOAP fault (code/message/detail), ham zarf (isteğe bağlı). SOAP 1.1/1.2, |
REST | Tüm HTTP metotları, JSON/metin/form gövde, OpenAPI/Swagger keşfi |
Kimlik doğrulama | Windows oturumu (Kerberos/NTLM, şifresiz), NTLM, basic, bearer, API key, WS-Security UsernameToken, istemci sertifikası |
Senaryolar | YAML; SOAP ve REST adımları aynı dosyada, |
Assertion'lar | Sunucu içinde, deterministik: |
Güvenlik | Host allowlist (WSDL import'ları ve yönlendirmeler dahil), prod ortam adları reddedilir, mutating operasyon engeli, maskeleme (TCKN checksum, IBAN, kart Luhn, telefon, e-posta, şifre/token alanları), JSONL denetim logu, şifreler hiçbir çıktıda görünmez |
Bağımsız kullanım |
|
Kurulum | Offline: Windows x64 ve x86 (32-bit) için Python 3.10–3.14 tekerlekleri zip içinde |
Neden EDR / DLP ajanlarını daha az tetikler?
Her servis çağrısında powershell.exe açılıp Invoke-WebRequest ile elle birleştirilmiş SOAP zarfları göndermek, script-block logging, AMSI ve EDR kurallarının tam olarak izlediği davranıştır. service-test-mcp ise VS Code'un bir kez başlattığı tek bir Python sürecidir; çağrı başına alt süreç açmaz, komut satırında URL, gövde veya şifre bulunmaz, bağlantılar yeniden kullanılır.
Dürüst not: Hiçbir araç, kurumunuzdaki güvenlik ürünlerinin hiçbir şey raporlamayacağını garanti edemez. Gerekirse BT ekibinizden yalnızca bu
service-test-mcp.exeyolu için istisna isteyin; bu, "her çağrıda PowerShell" istisnasından çok daha dar ve savunulabilir bir istisnadır.
Related MCP server: api-test-mcp
Kurulum (Windows, internet gerekmez)
Python'unuza uygun paketi indirip bir klasöre açın:
64-bit Python: service-test-mcp-simple-windows-x64.zip
32-bit Python: service-test-mcp-simple-windows-x86.zip
Python'unuzun kaç bit olduğunu
py -c "import struct; print(struct.calcsize('P')*8)"gösterir.Klasörü VS Code'da açın, Copilot Chat → Agent moduna şunu yazın:
KURULUM_REHBERI.md dosyasını oku ve "Ajan için kurulum adımları" bölümünü sırayla uygula.
Elle kurulum: zip klasöründe install.cmd çalıştırın → %USERPROFILE%\.service-test-mcp\Scripts\service-test-mcp.exe.
İnternet varsa:
py -m venv %USERPROFILE%\.service-test-mcp
%USERPROFILE%\.service-test-mcp\Scripts\pip install --prefer-binary "service-test-mcp-simple @ git+https://github.com/ohankay/service-test-mcp-simple.git"Hemen denemek için: demo servisler
Zip içindeki demo klasöründe WCF tarzı bir SOAP servisi ve bir REST API'ye karşı hazır services.yaml ve 3 senaryo var:
%USERPROFILE%\.service-test-mcp\Scripts\service-test-mcp.exe --demo-server
rem baska bir terminalde:
set STM_CONFIG=C:\Araclar\service-test-mcp-simple\demo\services.yaml
%USERPROFILE%\.service-test-mcp\Scripts\service-test-mcp.exe --run C:\Araclar\service-test-mcp-simple\demo\scenariosVS Code'a ekleme
Ctrl+Shift+P → MCP: Open User Configuration:
{
"inputs": [
{ "type": "promptString", "id": "stm-pw-ntlm", "description": "Servis kullanicisi sifresi", "password": true }
],
"servers": {
"service-test": {
"type": "stdio",
"command": "${userHome}\\.service-test-mcp\\Scripts\\service-test-mcp.exe",
"env": {
"STM_CONFIG": "${userHome}\\.service-test-mcp\\services.yaml",
"STM_PW_NTLM": "${input:stm-pw-ntlm}"
}
}
}
}Şifreyi VS Code bir kez sorar ve güvenli deposunda saklar. Hiçbir dosyaya düz metin olarak yazılmaz. Custom agent için .github/agents/service-tester.agent.md (tools: ['service-test/*', ...]), terminale kaçmayı önlemek için templates/copilot-instructions.md, hazır görevler için .github/prompts/ kullanılabilir.
Servis kataloğu (services.yaml)
default_environment: test
environments:
test:
verify_tls: true # veya kurum CA dosyası: C:\certs\kurum-ca.pem
allow_mutating: false
variables: { test_musteri_no: "1234567" } # senaryoda {{env.test_musteri_no}}
services:
HesapServisi:
protocol: soap
wsdl: { test: https://sunucu-test/HesapServisi.svc?wsdl }
auth: { type: windows } # şifresiz, oturum açmış Windows kullanıcısı
mutating_operations: [TransferYap, "*Sil"]
OdemeApi:
protocol: rest
base_url: { test: https://api-test.sirket.local/odeme/v1 }
openapi: { test: https://api-test.sirket.local/odeme/v1/swagger.json }
auth: { type: bearer, token_env: STM_TOKEN_API }Tüm seçenekler: bundle/templates/services.yaml. Bir servis hatalı tanımlıysa yalnızca o servis "broken" görünür, diğerleri çalışır. prod, production, prd, canli, live adlı ortamlar reddedilir. İstekler yalnızca o servisin o ortam için yapılandırılmış host'larına gidebilir; WSDL farklı bir adres yayınlıyorsa (endpoint veya allowed_hosts ile) açıkça izin verilmelidir.
Araçlar
Araç | Ne yapar |
| Servisler, ortamlar, varsayılan ortam, hatalı tanımlar |
| SOAP: port/binding, adres, operasyonlar. REST: base URL, OpenAPI operasyonları |
| Girdi/çıktı şeması, |
| Operasyon çağırmadan bağlantı testi (WSDL yükleme / base URL) |
| SOAP operasyonu çağırır; |
| REST çağrısı; |
|
|
| Son çağrının yanıtına assertion'ları uygular |
| Senaryo dosyası/klasörü/metni: yapı, servis adları, assertion sözdizimi, JSONPath, tanımsız değişkenler |
| Tek senaryo (dosya veya satır içi YAML) |
| Klasördeki tüm senaryolar, etiket filtresi, JUnit raporu |
|
|
Maskelenen değerler (***1326) modele geri yazdırılamaz; token ve id'ler extract ile saklanıp {{token}} olarak kullanılır. Assertion'lar ve extract her zaman gerçek değerlerle çalışır.
Senaryo formatı
id: login-bakiye
title: Login -> bakiye -> hesap detayı
tags: [smoke]
steps:
- name: Login
service: HesapServisi # SOAP: operation + params
operation: Login
params: { kullaniciAdi: "{{env.kullanici}}", sifre: "{{secret.STM_SECRET_TEST_SIFRE}}" }
assert:
- fault: none
- path: $.Token
notEmpty: true
extract: { token: $.Token, musteriNo: $.MusteriNo }
- name: Bakiye
service: HesapServisi
operation: GetBakiye
params: { token: "{{token}}", musteriNo: "{{musteriNo}}" }
assert:
- path: $.ToplamBakiyeTRY
numeric: { gte: 0 }
- path: $.Hesaplar.Hesap[*].Iban
matches: "^TR\\d{24}$"
all: true
- name: Hesap detayı
service: OdemeApi # REST: method + path (+ query, headers, body)
method: GET
path: /hesaplar/{{musteriNo}}
headers: { Authorization: "Bearer {{token}}" }
assert:
- status: 200
- schema: schemas/hesap.jsonAssertion referansı: fault: none | any | {code, message_contains}, status: 200 | [200, 201] | {gte: 200, lt: 300}, responseTimeMs: {lt: 2000}, header: Content-Type + contains, schema: dosya.json, ve path: <JSONPath> + bir operatör: equals, notEquals, equalsIgnoreCase, contains, notContains, startsWith, endsWith, matches, in, notIn, exists, notEmpty, isNull, type (string, number, integer, decimal, boolean, object, array, null, date, datetime), numeric (eq, ne, gt, gte, lt, lte, between, tolerance), count, length, schema. all: true her eşleşmeye uygular, message: hata metnine eklenir. Bir adım başarısız olursa sonraki adımlar atlanır (continue_on_failure: true hariç).
SOAP yanıtları JSON'a normalize edilir: namespace'ler düşer, maxOccurs > 1 elemanlar her zaman listedir, xsd:decimal Decimal olarak karşılaştırılır, tarihler ISO-8601 metindir.
VS Code'dan bağımsız kullanım
Hazır ajan (service-test-agent, ek paket gerekmez): OpenAI uyumlu herhangi bir chat-completions sunucusuna bağlanır, MCP sunucusunu kendisi başlatır.
set STM_AGENT_BASE_URL=http://qwen.sirket.local:8000/v1
set STM_AGENT_MODEL=qwen2.5-72b-instruct
set STM_CONFIG=%USERPROFILE%\.service-test-mcp\services.yaml
service-test-agent --check
service-test-agent --prompt "HesapServisi Login operasyonu icin pozitif ve negatif senaryo yaz ve kostur" --output rapor.md
service-test-agent &rem interaktif sohbetModel sunucusunda tool calling açık olmalı (vLLM + Qwen: --enable-auto-tool-choice --tool-call-parser hermes). Azure OpenAI için STM_AGENT_AZURE_API_VERSION, anahtar için STM_AGENT_API_KEY.
Kendi ajanınız / framework'ünüz: MCP sunucusunu stdio ile (service-test-mcp.exe) veya HTTP ile başlatın:
set STM_HTTP_TOKEN=uzun-rastgele-bir-deger
service-test-mcp --transport http --host 0.0.0.0 --port 8765Uç nokta http://sunucu:8765/mcp, istemci Authorization: Bearer <token> gönderir (yerel olmayan adreste token zorunludur). LangChain/LangGraph (langchain-mcp-adapters), Semantic Kernel, Claude Desktop ve diğer MCP istemcileri doğrudan bağlanır; allowlist, maskeleme, mutating engeli ve denetim logu her istemci için aynen geçerlidir.
AI'sız koşu (CI, zamanlanmış görev):
service-test-mcp --run D:\testler\senaryolar --env test --tags smoke --junit sonuc.xml --report sonuc.jsonÇıkış kodu: 0 hepsi geçti, 1 başarısız senaryo var, 2 yapılandırma/senaryo hatası.
Bilinmesi gerekenler
Desteklenmeyen WCF binding'leri:
netTcpBinding,netNamedPipeBindingve SecureConversation'lı mesaj güvenliği (wsHttpBindingvarsayılanı) Python'dan çağrılamaz. Test ortamında ek birbasicHttpBindingendpoint'i istemek en temiz yoldur. Birden çok port varsa SOAP 1.1 port'u tercih edilir;port_nameile seçilebilir.Maskeleme varsayılanları: TCKN (checksum ile), TR IBAN, kart numarası (Luhn ile), cep telefonu, e-posta ve şifre/token/secret/cvv/pin alanları. Müşteri ve hesap numaraları varsayılan olarak maskelenmez (test akışında gerekli);
masking.fieldsile eklenir,masking.unmask_fieldsile çıkarılır.Denetim logu:
%USERPROFILE%\.service-test-mcp\logs\audit-YYYYMMDD.jsonl; her çağrı için kullanıcı, servis, operasyon, ortam, durum, süre ve maskeli gövde.Çıktılar varsayılan olarak 20.000 karakterle sınırlıdır (
max_response_chars); büyük yanıtlarda assertion veyaextractkullanın.
Geliştirme
pip install -e ".[dev]"
pytest -v # demo servislere karşı birim, entegrasyon, stdio/HTTP e2e ve ajan testleri
pip install uv && python scripts/build_bundle.py # release/*.zip (x64 + x86, Python 3.10-3.14)PyPI'da 32-bit Python 3.14 için PyYAML tekerleği olmadığından vendor/ altında saf Python bir PyYAML tekerleği bulunur (PYYAML_FORCE_LIBYAML=0 pip wheel pyyaml==6.0.3); yalnızca ikili tekerlek olmayan hedefte kullanılır.
Lisans
MIT
Available Tools
12 toolscheck_last_responseARead-only
Evaluate assertions against the last soap_call / rest_call response (real values, deterministic). Use it to try assertions before writing them into a scenario file.
| Name | Required | Description | Default |
|---|---|---|---|
| assertions | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered. The description adds genuinely useful behavior beyond that: it evaluates against real values from the most recent call and is deterministic. It omits what happens when no prior call exists (error vs empty), which is the one meaningful gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core action and followed by the usage rule. Nothing is redundant or padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return values needn't be described. For a one-parameter tool this is nearly adequate, but the complete absence of assertion syntax in both schema and description leaves the caller guessing how to construct the required argument.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the single 'assertions' parameter is an untyped array of free-form objects with additionalProperties=true. The description says 'assertions' but never describes their structure, keys, or supported operators — the one thing an agent must know to call this correctly. It clarifies only the evaluation target (the last response).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: evaluate assertions against the last soap_call / rest_call response. Naming the two source tools and the 'last response' scope makes the purpose unmistakable and separates it from suite/scenario runners. It stops short of explicitly contrasting with validate_scenario, which is the nearest sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit when-to-use: try assertions here before writing them into a scenario file. That implicitly names the alternative workflow (run_scenario / validate_scenario) and the condition that selects this tool. No explicit when-not-to-use is given, but the framing is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
check_serviceBRead-only
Connectivity check without calling an operation: SOAP loads the WSDL and binds the port; REST requests base_url. Without 'service' every service is checked.
| Name | Required | Description | Default |
|---|---|---|---|
| service | No | ||
| environment | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered; the description adds real value by disclosing the side-effect-free mechanics (WSDL fetch + port binding vs. base_url request) and the null-service default of checking everything. It stops short of saying what the check returns on failure or whether it retries, but with annotations present the bar is lower.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the core definition front-loaded and the default-scope rule following. 'REST requests base_url' is compressed almost to the point of ambiguity (GET? HEAD? timeout?), which is the only thing keeping it from a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, and the mechanics are covered. But with 0% schema coverage, the undocumented 'environment' parameter is a real completeness hole for a tool whose whole job is scoped connectivity testing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must carry the load, yet it only explains one of two parameters: 'service' (omitted = check all). The 'environment' parameter is never mentioned anywhere, leaving an agent to guess what values it accepts or what it scopes. This is a clear compensation gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('connectivity check') and immediately scopes it as 'without calling an operation', which separates it from soap_call/rest_call siblings. It also explains the per-protocol mechanics (SOAP loads WSDL and binds port, REST hits base_url), so an agent knows exactly what kind of check this is. It never names a sibling explicitly, keeping it short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Without calling an operation' implicitly tells the agent to use this for validation rather than invocation, and 'Without service every service is checked' clarifies the sweep behavior. However, it never states when to prefer this over just calling soap_call/rest_call, nor any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
describe_operationBRead-only
Input/output schema of an operation. SOAP: operation name -> input_schema, example_params, output_schema. REST: 'METHOD /path' or operationId from the OpenAPI document.
| Name | Required | Description | Default |
|---|---|---|---|
| service | Yes | ||
| operation | Yes | ||
| environment | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Read-only and closed-world hints are already declared in annotations, so the safety profile is covered. The description adds value by enumerating what SOAP returns (input_schema, example_params, output_schema). It does not mention error behavior for invalid operations or environment scoping, so it only modestly exceeds annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the purpose front-loaded and format variants packed in without filler. Dense and efficient, though the SOAP/REST formatting could be marginally clearer in structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return-value detail need not be explained, and the operation-param format is covered. But for a three-parameter tool at 0% schema coverage, the meaning of service and environment (and how environment affects the lookup) is left entirely to inference, which is a real gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description carries the burden. It gives concrete format guidance for the operation parameter (SOAP operation name, or REST 'METHOD /path' / operationId from the OpenAPI document), which is genuinely useful. However, it says nothing about the service or environment parameters, leaving two of three undocumented.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear purpose: returns the input/output schema of an operation, and characterizes the resource as an operation rather than a service, distinguishing it from siblings like describe_service. The SOAP-vs-REST formats for the operation identifier are spelled out. It stops short of explicitly naming competing siblings, but an agent can tell what it does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the format of the operation parameter but gives no guidance on when to reach for this tool versus describe_service, check_service, or the call tools. There are no prerequisites or exclusions; the agent must infer that this is the tool for inspecting an operation's schema before invoking it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
describe_serviceCRead-only
SOAP: WSDL ports, selected address and operation names. REST: base_url and OpenAPI operations (if configured).
| Name | Required | Description | Default |
|---|---|---|---|
| service | Yes | ||
| environment | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so safety is covered. The description usefully discloses that output varies by protocol (WSDL ports/address/operations vs base_url/OpenAPI operations), but says nothing about missing services, environment resolution, or error behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is two short clauses with no wasted words, but it is a sentence fragment with no main clause stating the tool's action, so it is terse at the cost of clarity rather than effectively front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, yet the description spends its only content on returns. For a two-parameter tool with zero schema coverage, it leaves the purpose, parameter semantics, and sibling differentiation unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description never mentions the required 'service' parameter or the optional 'environment' parameter. The SOAP/REST distinction hints that the service determines output shape, but neither parameter's meaning or format is clarified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description never states a verb or that it returns metadata about a named service; it only lists the fields returned for SOAP and REST. An agent can infer it describes a service, but it does not distinguish this from describe_operation or list_services.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this instead of list_services, describe_operation, check_service, or the call tools. The only conditional, 'if configured,' concerns OpenAPI output, not tool selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_servicesBRead-only
List configured services (SOAP/REST), environments and the default environment. Start here.
| Name | Required | Description | Default |
|---|---|---|---|
| environment | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish readOnlyHint=true and openWorldHint=false, so safety and locality are covered. The description adds only a hint about returned content (services plus environments), which the output schema already conveys; it says nothing about auth needs, result size, or whether the call hits a remote source despite openWorldHint=false.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with the enumeration scope front-loaded and the sequencing cue last; nothing is wasted or buried.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, and annotations cover the read-only profile. However, for a discovery tool with one undocumented filter parameter, the description should still clarify the environment argument — that gap keeps it at minimum-viable completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single 'environment' parameter has 0% schema description coverage, and the description only says environments are listed — it never explains whether passing 'environment' filters the list, what the default null means, or what string format is expected. This leaves the one parameter's semantics unresolvable from the definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and the exact resources returned: services (SOAP/REST), environments, and the default environment. It stops short of contrasting with describe_service or other siblings, but an agent can tell this is the enumeration tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Start here." gives a weak but real sequencing cue, implying this is the entry point before describe_service/describe_operation/check_service. There is no explicit when-not guidance or named alternative, so usage is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
reload_configARead-only
Re-read services.yaml after it was edited (no server restart needed).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false, so the safety profile is covered; the description adds the useful context that the reload is hot (no restart). It does not say what happens if services.yaml is malformed, whether the reload is atomic, or whether partial configs can be applied — meaningful gaps for a reload operation, though mitigated by the annotation coverage and the output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the action and target file, followed by the condition and the no-restart benefit. Nothing is wasted and nothing extraneous is included.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with an output schema (so return values need no explanation) and read-only annotations, the description covers what the agent needs to decide to call it. The only missing piece is failure/validation behavior when the edited file is invalid.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline is 4. Schema coverage is 100% and there is nothing parameter-wise the description needs to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Re-read') and a specific resource ('services.yaml'), which no sibling tool touches (siblings are list_services, describe_service, check_service, run_scenario, etc.). An agent can identify this as the config-reload action without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It names an explicit trigger condition: use it after services.yaml has been edited, and states the key benefit (no server restart needed). It does not name an alternative or exclusion, but for a config-reload tool there is no plausible competing sibling, so 4 is appropriate rather than 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
rest_callA
Call a REST endpoint. path is relative to the service base_url (e.g. /hesaplar/123). body = JSON value, body_text = raw text (XML, form). path, query, headers and body may use {{variables}} (e.g. headers {"Authorization": "Bearer {{token}}"}). extract = {name: JSONPath} saves session variables. Returns status, body (parsed JSON or text), elapsed_ms.
| Name | Required | Description | Default |
|---|---|---|---|
| body | No | ||
| path | Yes | ||
| query | No | ||
| method | Yes | ||
| extract | No | ||
| headers | No | ||
| service | Yes | ||
| body_text | No | ||
| environment | No | ||
| include_headers | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations only say readOnly=false, openWorld=true, destructive=false, so the description carries the real burden and does well: it discloses that {{variables}} are interpolated in path/query/headers/body, that extract writes values into session variables (a side effect beyond annotations), and what the response contains (status, parsed body, elapsed_ms). It omits error/failure behavior, whether redirects or retries occur, and what happens when a variable is undefined.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Five dense lines, front-loaded with the core action, then parameter semantics, then return shape; nothing is padding. Minor cost: the shorthand 'x = y' formatting and the trailing return-shape sentence (redundant with the output schema) reduce scannability slightly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 10-parameter, 0%-coverage, open-world HTTP tool, the description covers the tricky parts (templating, extract, body vs body_text) but leaves environment, include_headers, and method/service usage unexplained, and gives no routing guidance against soap_call or the describe_* discovery tools. Adequate but with clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% (only bare titles), so the description must compensate, and it explains path, body vs body_text, extract's JSONPath map, and templating in query/headers with a concrete Authorization example. However, four parameters are never addressed: service, method, environment (which environment's base_url and credentials — a meaningful behavior), and include_headers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear verb+resource ('Call a REST endpoint') and clarifies that the path is relative to a named service's base_url, which distinguishes it implicitly from soap_call. It never explicitly names siblings like describe_operation or list_services, so an agent must infer the REST-vs-SOAP and discover-then-call split.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the base_url/service relationship and the extract-to-session-variable mechanism hint at a multi-step workflow (describe_service -> rest_call -> reuse variables). There is no explicit when-to-use vs soap_call, no warning about calling before describe_operation, and no guidance on which HTTP methods are safe.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_scenarioA
Run one scenario file (or inline YAML 'content'). Steps run in order; extract passes values to later steps. verbosity: summary | failures (default: details only for failed steps) | full.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | ||
| content | No | ||
| variables | No | ||
| verbosity | No | failures | |
| environment | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds real behavior beyond the annotations: steps execute sequentially, `extract` propagates values to later steps, and the three verbosity modes are defined with their default and what each shows. It does not say what happens when a step fails (abort vs continue) or disclose the external side effects implied by openWorldHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences, front-loaded with the action and target, followed by execution semantics and the verbosity legend. The verbosity enumeration is terse to the point of being slightly hard to parse, but no sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, and verbosity semantics are covered. However, for a mutating, open-world executor with five undocumented parameters, the description omits failure handling, side-effect scope, and two of five parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description must carry all parameter meaning; it partially does by explaining path vs inline `content` and enumerating the verbosity values. `variables` (how values reach step interpolation) and `environment` are left completely undocumented, and the mutual exclusivity/optionality of path and content is only implied.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ("Run one scenario file (or inline YAML 'content')") and clarifies the unit is a single scenario, which hints at the distinction from run_test_suite. It never names a sibling, so an agent must infer how it differs from validate_scenario.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the tool obviously executes scenarios rather than validating them, but the description never says when to prefer it over validate_scenario or run_test_suite, nor states prerequisites (e.g., that a scenario should be validated first, or that steps hit external services).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_test_suiteA
Run every scenario under a folder (recursively), optionally only those with one of 'tags'. Returns a summary and failed assertions; junit_path writes a JUnit XML report.
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | ||
| tags | No | ||
| verbosity | No | summary | |
| junit_path | No | ||
| environment | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With annotations already declaring readOnlyHint=false, openWorldHint=true, destructiveHint=false, the description adds real value: it discloses the return content (summary and failed assertions) and the side effect of junit_path writing a JUnit XML file. It does not mention that execution may hit external services or any permission/environment implications, so it stops short of 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences, front-loaded with the core action and scope, then the filtering option and side-effect. Every clause carries information; only minor cost is the awkward line break splitting the sentence.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be explained, though the description usefully mentions failures. However, with five parameters at zero schema documentation, two parameters (verbosity, environment) remain entirely opaque, leaving the definition incomplete for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must carry parameter meaning. It explains path (recursive folder traversal), tags (selective filtering), and junit_path (report output), but leaves verbosity and environment completely undocumented in either schema or description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Run every scenario under a folder (recursively)') and its scope distinguishes it from the single-scenario sibling run_scenario. It does not name that sibling explicitly, but the folder-level scope makes the difference inferrable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it (bulk folder execution, optional tag filtering) but never states when to prefer run_scenario or validate_scenario instead, nor any preconditions. Usage is inferable rather than guided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
session_variablesBRead-only
Show (masked) or clear the session variables saved by 'extract' on soap_call / rest_call.
| Name | Required | Description | Default |
|---|---|---|---|
| clear | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description advertises a 'clear' mode that destroys saved session state, while annotations declare readOnlyHint=true. An agent trusting the annotation could call with clear=true believing the operation is side-effect free, so the description and structured metadata conflict on the safety profile. The masking detail is useful, but the contradiction dominates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence, front-loaded with the primary action, that names both operating modes and the source of the data with zero filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, and the masking note covers a key concern. But for a tool with a state-destroying mode, the description says nothing about whether clearing is permanent, session-scoped, or what remains afterward, leaving a meaningful gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With one parameter at 0% schema coverage, the description carries the burden and does so: it distinguishes the two modes (default show-and-mask vs. clear) that map directly onto the boolean 'clear' flag. It stops short of stating that clear=true is destructive/irreversible or that masking is the default when the flag is omitted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description pairs two concrete verbs (show/clear) with a specific resource (session variables) and traces their origin to the 'extract' feature of soap_call/rest_call, which no sibling tool covers. An agent can tell what this does without opening the schema, though it never explicitly contrasts itself with check_last_response or the call tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The mention of variables 'saved by extract on soap_call / rest_call' implies the situation in which an agent would want this tool (inspecting values captured from a prior call). However, there is no explicit statement of when to prefer it over check_last_response, nor any prerequisite such as 'call soap_call with extract first' stated as a condition.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
soap_callA
Call a SOAP/WCF operation. params = JSON object of the operation's input elements (see describe_operation). Values may use {{variables}}. extract = {name: JSONPath} saves values from the response as session variables (e.g. {"token": "$.Token"}) so later calls can send {{token}} even though it is masked in the output. Returns status, fault (code/message/detail) or body (JSON), elapsed_ms. include_raw adds the SOAP envelopes.
| Name | Required | Description | Default |
|---|---|---|---|
| params | No | ||
| extract | No | ||
| service | Yes | ||
| operation | Yes | ||
| environment | No | ||
| include_raw | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by disclosing real behavioral traits: {{variable}} substitution in inputs, the extract mechanism that persists values as session variables across calls, the fact extracted values are masked in output, and the return shape (status, fault code/message/detail, body, elapsed_ms). Those masking and side-effect details are precisely what an agent cannot infer from readOnlyHint=false/openWorldHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core action, then packs the input-shaping rules and return contract into short parenthetical clauses with no filler. It is dense but every sentence carries distinct information; the one cost is that the extract/masking sentence is a long run-on.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, yet the description still summarizes the return contract and the fault shape, which is useful for error handling. The remaining gap is the environment parameter and any guidance on authentication or failure recovery, which an agent would still have to discover elsewhere.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must carry parameter meaning, and it covers only three of six: params, extract (with a concrete JSONPath example) and include_raw. service, operation and especially environment — which is optional and non-obvious — are left entirely unexplained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Call a SOAP/WCF operation") and immediately scopes the two key inputs to that resource, so an agent can tell this apart from rest_call. It stops short of naming the REST sibling explicitly as the alternative, leaving a small amount of inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Points to describe_operation as the prerequisite for knowing the operation's input elements, which gives a clear workflow context for correct invocation. It does not state when to prefer this over rest_call or when this tool is inappropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
validate_scenarioARead-only
Validate scenario YAML (a file, a folder, or inline 'content'): structure, services, assertion syntax, JSONPath, and variables used before they are defined. Does not call any service.
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | ||
| content | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and openWorldHint=false; the description reinforces this with the concrete claim that no service is called, which is meaningful additional context for a validator. It stops short of saying what a failure looks like, but an output schema exists to cover that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with the input forms in parentheses and a short scoping clause. Nothing is redundant and the enumeration of checks is compact.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return values need not be explained, and the description covers inputs and scope of validation adequately. The remaining gap is the optional-parameter contract (behavior when both path and content are omitted), which the description leaves silent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description has to carry the weight, and it does: 'path' is clarified as accepting a file OR a folder (real meaning the schema lacks), and 'content' as the inline alternative. It does not state that at least one of the two must be supplied, despite zero required parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Validate') and resource ('scenario YAML'), then enumerates the checks performed: structure, services, assertion syntax, JSONPath, and variable ordering. 'Does not call any service' implicitly separates it from run_scenario, though it never names that sibling, so an agent must infer the pairing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The accepted input shapes (file, folder, inline content) and the no-side-effect nature imply a pre-flight/static-check usage, but there is no explicit 'use this before run_scenario' or statement of when-not to use it. The agent can infer the workflow but is not routed to it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
12 tool updates
v1.0.0- First observed
check_last_response - First observed
check_service - First observed
describe_operation - First observed
describe_service - First observed
list_services - First observed
reload_config - First observed
rest_call - First observed
run_scenario - First observed
run_test_suite - First observed
session_variables - First observed
soap_call - First observed
validate_scenario
TDQS
Scored across 12 tools
Most tools have clearly distinct roles: discovery (list_services), introspection (describe_service vs describe_operation differ by granularity), connectivity (check_service), and invocation split cleanly by protocol (soap_call vs rest_call). The main soft overlap is run_scenario vs run_test_suite (single file vs folder/tags) and check_last_response vs scenario assertions, but descriptions make the boundaries explicit enough to choose correctly.
The set is predominantly consistent snake_case verb_noun (list_services, describe_operation, check_service, validate_scenario, run_scenario, reload_config). Minor deviations exist: soap_call/rest_call use protocol-first ordering and session_variables is a noun-only name with no verb, but the pattern remains readable and predictable.
At 12 tools the server is well-scoped for an API/service testing surface, with each tool mapping to a distinct phase (discover, describe, check, call, validate, run, inspect, reload). Nothing feels redundant or missing at the count level.
The lifecycle is largely covered: discovery, schema introspection, connectivity checks, SOAP/REST invocation with variable extraction, assertion probing, scenario validation and execution (single and suite), plus config reload. Gaps are minor: no tool to list/discover available scenario files or suites, and no built-in scenario authoring/write operation, so agents must rely on external file access.
Maintenance
Related MCP Connectors
AI-callable tools for API mocking, testing, monitoring, security, and automation.
Drive OctoPerf load testing from any AI agent: import, edit, validate, run scenarios, read metrics.
Webhook, email, WebSocket and tunnel testing for AI agents: capture, wait, verify, replay.
Free AI test helpers: injection inputs, OWASP mapping, release plans, response and tool-call packs.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to inspect, test, validate, compare, and debug HTTP APIs through a structured toolkit, with features such as secret redaction, response contract validation, regression comparison, health checks, and incident investigation workflows.MIT
- AlicenseAqualityCmaintenanceEnables AI agents to call real API endpoints and verify responses against OpenAPI specs, supporting contract testing, auth presets, spec diffing, and health checks.80MIT
- AlicenseCqualityAmaintenanceEnables AI agents to safely call legacy SOAP/WSDL systems as typed, schema-validated MCP tools, with write operations excluded by default and every call audit-logged.3862 PyPI6Apache 2.0
- AlicenseBqualityBmaintenanceEnables AI agents to drive API testing from YAML definitions, executing and chaining requests without hand-written curl, while keeping credentials out of the conversation and responses token-cheap, plus a VANDER-based QA review of endpoints.21MIT