SpecProof

Generate Conformance Tests

generate_tests

Read-only

Generate a cited, runnable conformance test that verifies one normative requirement.

The test is grounded in the requirement sentence plus its surrounding clause from the
corpus, and every test cites the spec, section and page it enforces. If the requirement
is not testable (boilerplate, scope text), the generator abstains rather than inventing
a test.

Args:
    requirement_id: The requirement's id, as shown by get_requirements
    target: "pytest" for a runnable Python test module, or "gherkin" for a reviewable
            .feature file (default: pytest)

Input Schema

TableJSON Schema

Name	Required	Description	Default
`target`	No		pytest
`requirement_id`	Yes

Output Schema

TableJSON Schema

Name	Required	Description	Default
`result`	Yes

TDQS

A4.6/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, and the description adds meaningful behavioral context: the test is grounded in the requirement sentence and clause, cites spec/section/page, and the generator abstains rather than inventing tests. This goes beyond annotations though it does not detail failure modes or return values, which are covered by the output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core purpose, followed by a concise behavioral paragraph and clear Args documentation. Every sentence adds value without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 2 parameters and an existing output schema, the description covers input semantics, the abstention behavior, citation grounding, and target choices. It is complete enough for an agent to select and invoke the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but the description fully compensates by explaining requirement_id as the id shown by get_requirements and target as 'pytest' for a runnable module or 'gherkin' for a reviewable .feature file with default pytest. This adds substantial meaning beyond the raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool 'generates a cited, runnable conformance test' for 'one normative requirement', identifying the specific verb and resource. It distinguishes itself from siblings by referencing requirement_id from get_requirements and offering distinct target formats (pytest/gherkin).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clear usage context is provided: the requirement_id is sourced from get_requirements, and the target parameter selects between pytest and gherkin. It also describes abstention behavior for untestable requirements, but does not explicitly contrast with alternative sibling tools or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All SpecProof tools Inspect

Try in Browser

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A3.8/5.0

Disambiguation4/5

Tools are largely distinct but there are clusters of similar functionality, such as multiple search mechanisms (search_specifications, semantic_search, get_requirements with a search parameter) and reference-graph tools (get_spec_references, get_spec_dependents, get_relationship_graph). Descriptions do differentiate them, but an agent could still be uncertain which to use for a given query.

Naming Consistency4/5

Most tool names follow a snake_case verb_noun pattern (e.g., list_documents, search_specifications, get_requirements). However, semantic_search is adjective_noun rather than verb_noun, and get_database_stats is a generic outlier among the spec-focused tools. Overall the pattern is consistent with minor deviations.

Tool Count4/5

With 17 tools, the set is slightly above the ideal 3-15 range but not excessive for the breadth of functionality: search, metadata, requirements, test generation, relationships, and schemas. Each tool has a defined role, and the count feels warranted for the domain.

Completeness4/5

The tool set covers the core workflow well: discovering specs (search_specifications, list_documents), retrieving content (get_document_content, get_requirements), analyzing relationships (get_spec_references, get_spec_dependents), and generating tests (build_cross_spec_suite, generate_tests). Minor gaps exist, such as the lack of a direct tool to fetch a requirement's full surrounding clause (workaround via get_document_content), but no critical dead ends.

Resources

Need Help?