# MCP Response Schema Guide
This document captures the canonical response contract that every Foundry MCP operation must follow. It complements the completed specification [`specs/completed/response-schema-standardization-2025-11-26-001.json`](../../specs/completed/response-schema-standardization-2025-11-26-001.json), the shared helpers in [`src/foundry_mcp/core/responses.py`](../../src/foundry_mcp/core/responses.py), and the best-practice guidance in [dev_docs/mcp_best_practices](../mcp_best_practices/README.md).
## Standard Envelope
All tool responses **must** serialize to the following structure ("response-v2"):
```json
{
"success": true,
"data": { ... },
"error": null,
"meta": {
"version": "response-v2",
"request_id": "req_abc123"?,
"warnings": ["..."]?,
"warning_details": [{ "code": "...", "severity": "...", "message": "..." }]?,
"pagination": { ... }?,
"rate_limit": { ... }?,
"telemetry": { ... }?,
"content_fidelity": "full" | "partial" | "summary" | "reference_only"?,
"content_fidelity_schema_version": "1.0"?,
"dropped_content_ids": ["..."]?,
"content_archive_hashes": { ... }?
}
}
```
- `success` (bool): Indicates whether the operation completed successfully.
- `data` (object): Operation payload. When no payload exists, send `{}`. For errors, may contain structured error context (see [Error Response Fields](#error-response-fields)).
- `error` (string | null): Populated only when `success` is `false`. Human-readable error description.
- `meta` (object): Always include `{"version": "response-v2"}` and attach optional metadata using the reserved keys listed below.
### Metadata Field Semantics
| Key | Required | Description |
|------------------|----------|-------------|
| `version` | YES | Identifies the response schema version (`response-v2`). |
| `request_id` | SHOULD | Correlation identifier propagated through logs/traces. |
| `warnings` | SHOULD | Non-fatal issues for successful operations (array of strings). |
| `pagination` | MAY | Cursor-based pagination object containing `cursor`, `has_more`, `total_count`, etc. |
| `rate_limit` | MAY | Remaining quota, reset time, and retry hints when throttling occurs. |
| `telemetry` | MAY | Timing/performance metrics such as `duration_ms` or downstream call counts. |
| `content_fidelity` | MAY | Content fidelity level indicating completeness of the response (see [Content Fidelity Metadata](#content-fidelity-metadata)). |
| `content_fidelity_schema_version` | MAY | Schema version for content fidelity metadata (e.g., `"1.0"`). |
| `dropped_content_ids` | MAY | Array of content identifiers that were dropped due to size constraints. |
| `content_archive_hashes` | MAY | Object mapping archive identifiers to content hashes for retrieval. |
| `warning_details` | MAY | Structured warning objects with severity and context (see [Warning Details](#warning-details)). |
> Do **not** invent new top-level keys under `data` to convey metadata. Attach operational context through `meta` so every tool shares the same envelope semantics.
### Edge Case Semantics
| Scenario | Required Behavior |
|----------------------------|--------------------|
| Empty but successful query | `success: true`, include empty arrays/counts in `data`, `error: null`. |
| Missing resource / invalid input | `success: false`, `data` contains structured error info, descriptive `error` string. |
| Blocked or partial work | `success: true`, describe state inside `data`, add `meta.warnings` if applicable. |
| Multi-payload operations | Nest each payload under a named key inside `data` (e.g., `{ "spec": {...}, "tasks": [...] }`). |
## Content Fidelity Metadata
When responses may be truncated, summarized, or have content dropped due to token limits or size constraints, include content fidelity metadata in `meta` to inform consumers about response completeness.
### Content Fidelity Schema
```json
{
"meta": {
"version": "response-v2",
"content_fidelity_schema_version": "1.0",
"content_fidelity": "full" | "partial" | "summary" | "reference_only",
"dropped_content_ids": ["finding-003", "source-015"],
"content_archive_hashes": {
"archive-001": "sha256:abc123..."
}
}
}
```
### Content Fidelity Field Definitions
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `content_fidelity_schema_version` | string | SHOULD (when fidelity < full) | Schema version for content fidelity metadata. Current version: `"1.0"`. |
| `content_fidelity` | string | SHOULD (when fidelity < full) | Level of content completeness in the response. |
| `dropped_content_ids` | array\<string\> | MAY | Identifiers of content items that were omitted. Enables targeted retrieval. |
| `content_archive_hashes` | object | MAY | Map of archive IDs to content hashes for retrieving dropped content. |
### Content Fidelity Levels
| Level | Description | Use Case |
|-------|-------------|----------|
| `full` | Complete response with all content included | Default when no truncation occurs |
| `partial` | Some content omitted but structure preserved | Large responses exceeding soft limits |
| `summary` | Condensed representation of full content | Token-constrained contexts |
| `reference_only` | Only identifiers/references, no content bodies | Extreme token constraints |
### Content Fidelity Example
Response with partial fidelity due to dropped findings:
```json
{
"success": true,
"data": {
"research_id": "research-001",
"findings": [
{"id": "finding-001", "title": "Primary result", "content": "..."},
{"id": "finding-002", "title": "Secondary result", "content": "..."}
],
"total_findings": 5
},
"error": null,
"meta": {
"version": "response-v2",
"content_fidelity_schema_version": "1.0",
"content_fidelity": "partial",
"dropped_content_ids": ["finding-003", "finding-004", "finding-005"],
"content_archive_hashes": {
"findings-archive": "sha256:e3b0c44298fc1c149afbf4c8996fb924..."
},
"warnings": ["3 findings omitted due to token limits"]
}
}
```
### Content Retrieval Pattern
When `dropped_content_ids` is present, consumers can retrieve omitted content:
1. Check `dropped_content_ids` for missing item identifiers
2. Use `content_archive_hashes` to verify archive availability
3. Call appropriate retrieval endpoint with the archive hash or content IDs
## Warning Details
For structured warnings with severity and context beyond simple string messages, use `warning_details` alongside or instead of the `warnings` array.
### Warning Details Schema
```json
{
"meta": {
"version": "response-v2",
"warnings": ["3 findings omitted due to token limits"],
"warning_details": [
{
"code": "CONTENT_TRUNCATED",
"severity": "info",
"message": "3 findings omitted due to token limits",
"context": {
"dropped_count": 3,
"total_count": 5,
"reason": "token_limit_exceeded"
}
}
]
}
}
```
### Warning Detail Field Definitions
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `code` | string | SHOULD | Machine-readable warning classification (e.g., `CONTENT_TRUNCATED`, `STALE_CACHE`). |
| `severity` | string | SHOULD | Warning severity level: `info`, `warning`, `error`. |
| `message` | string | YES | Human-readable warning description. |
| `context` | object | MAY | Additional context specific to the warning type. |
### Warning Severity Levels
| Severity | Description | Consumer Action |
|----------|-------------|-----------------|
| `info` | Informational, no action needed | Log/display as appropriate |
| `warning` | Potential issue, consider action | Evaluate context and decide |
| `error` | Significant issue, action recommended | Address before proceeding |
### Standard Warning Codes
| Code | Severity | Description |
|------|----------|-------------|
| `CONTENT_TRUNCATED` | info | Response content was truncated due to size limits |
| `STALE_CACHE` | warning | Cached data may be outdated |
| `PARTIAL_FAILURE` | warning | Some sub-operations failed but overall succeeded |
| `DEPRECATED_FIELD` | info | Response includes deprecated fields |
| `RATE_LIMIT_APPROACHING` | warning | Approaching rate limit threshold |
| `FALLBACK_USED` | info | Primary source unavailable, fallback used |
### Warning Details Example
```json
{
"success": true,
"data": {
"results": [...]
},
"error": null,
"meta": {
"version": "response-v2",
"warnings": [
"3 sources failed to respond",
"Cache data is 2 hours old"
],
"warning_details": [
{
"code": "PARTIAL_FAILURE",
"severity": "warning",
"message": "3 sources failed to respond",
"context": {
"failed_sources": ["source-a", "source-b", "source-c"],
"successful_sources": 7,
"total_sources": 10
}
},
{
"code": "STALE_CACHE",
"severity": "warning",
"message": "Cache data is 2 hours old",
"context": {
"cache_age_seconds": 7200,
"max_freshness_seconds": 3600
}
}
]
}
}
```
## Error Response Fields
When `success` is `false`, the `data` object should contain structured error context to enable machine-readable error handling. The `error_response` helper automatically populates these fields.
### Standard Error Fields
| Field | Required | Type | Description |
|-------|----------|------|-------------|
| `error_code` | SHOULD | string | Machine-readable error classification (e.g., `VALIDATION_ERROR`, `NOT_FOUND`, `RATE_LIMIT_EXCEEDED`). Use SCREAMING_SNAKE_CASE. |
| `error_type` | SHOULD | string | Error category for routing/handling (e.g., `validation`, `authorization`, `not_found`, `internal`). |
| `remediation` | SHOULD | string | Actionable guidance for resolving the error. |
| `details` | MAY | object | Nested structure with field-specific or context-specific error info. |
### Error Type Categories
| error_type | HTTP Analog | Description | Retry? |
|------------|-------------|-------------|--------|
| `validation` | 400 | Invalid input data | No, fix input |
| `authentication` | 401 | Invalid or missing credentials | No, re-authenticate |
| `authorization` | 403 | Insufficient permissions | No |
| `not_found` | 404 | Requested resource doesn't exist | No |
| `conflict` | 409 | State conflict (e.g., duplicate) | Maybe, check state |
| `rate_limit` | 429 | Too many requests | Yes, after delay |
| `feature_flag` | 403 | Feature not enabled for client | No, check flag status |
| `internal` | 500 | Server-side error | Yes, with backoff |
| `unavailable` | 503 | Service temporarily unavailable | Yes, with backoff |
### Error Response Example
```json
{
"success": false,
"data": {
"error_code": "VALIDATION_ERROR",
"error_type": "validation",
"remediation": "Provide a non-empty spec_id parameter",
"details": {
"field": "spec_id",
"constraint": "required",
"received": null
}
},
"error": "Validation failed: spec_id is required",
"meta": {
"version": "response-v2",
"request_id": "req_abc123"
}
}
```
### Common Error Codes
| Error Code | error_type | When to Use |
|------------|------------|-------------|
| `VALIDATION_ERROR` | validation | Generic input validation failure |
| `INVALID_FORMAT` | validation | Malformed input (wrong type, bad JSON) |
| `MISSING_REQUIRED` | validation | Required field not provided |
| `NOT_FOUND` | not_found | Resource doesn't exist |
| `SPEC_NOT_FOUND` | not_found | Specification file not found |
| `TASK_NOT_FOUND` | not_found | Task ID not found in spec |
| `DUPLICATE_ENTRY` | conflict | Resource already exists |
| `CONFLICT` | conflict | State conflict or invalid transition |
| `UNAUTHORIZED` | authentication | Invalid or missing credentials |
| `FORBIDDEN` | authorization | Insufficient permissions |
| `FEATURE_DISABLED` | feature_flag | Feature flag not enabled |
| `RATE_LIMIT_EXCEEDED` | rate_limit | Too many requests |
| `INTERNAL_ERROR` | internal | Unexpected server error |
| `UNAVAILABLE` | unavailable | Service temporarily unavailable |
## Helper Usage
Always leverage the shared helpers and dataclasses to produce responses:
```python
from dataclasses import asdict
from foundry_mcp.core.responses import success_response, error_response
@mcp.tool()
def tool_example(...) -> dict:
payload = compute_payload(...)
return asdict(success_response(
data={"result": payload},
warnings=payload.warnings,
pagination=payload.pagination,
request_id=context.request_id,
))
```
### Error Response Helper
For failures, use `error_response()` to include machine-readable context:
```python
return asdict(error_response(
message="Validation failed: spec_id is required",
error_code="MISSING_REQUIRED",
error_type="validation",
remediation="Provide a non-empty spec_id parameter",
details={"field": "spec_id", "constraint": "required"},
request_id=context.request_id,
))
```
**`error_response()` Parameters:**
| Parameter | Required | Description |
|-----------|----------|-------------|
| `message` | YES | Human-readable error description (populates `error` field) |
| `error_code` | SHOULD | Machine-readable code (e.g., `VALIDATION_ERROR`) |
| `error_type` | SHOULD | Error category (e.g., `validation`, `not_found`) |
| `remediation` | SHOULD | Actionable guidance for resolving the error |
| `data` | MAY | Additional context to merge into `data` object |
| `details` | MAY | Nested error details (field info, constraints) |
| `request_id` | SHOULD | Correlation ID for tracing |
| `rate_limit` | MAY | Rate limit state when applicable |
| `telemetry` | MAY | Timing/performance data captured before failure |
| `meta` | MAY | Additional metadata to merge into `meta` object |
**Example: Not Found Error**
```python
return asdict(error_response(
message=f"Spec '{spec_id}' not found",
error_code="SPEC_NOT_FOUND",
error_type="not_found",
remediation="Verify the spec ID exists using spec(action=\"list\")",
request_id=context.request_id,
))
```
**Example: Feature Flag Disabled**
```python
return asdict(error_response(
message=f"Feature '{flag_name}' is not enabled",
error_code="FEATURE_DISABLED",
error_type="feature_flag",
data={"feature": flag_name},
remediation="Contact support to enable this feature or check feature flag configuration",
))
```
**Example: Rate Limit Exceeded**
```python
return asdict(error_response(
message="Rate limit exceeded: 100 requests per minute",
error_code="RATE_LIMIT_EXCEEDED",
error_type="rate_limit",
data={"retry_after_seconds": 45},
remediation="Wait 45 seconds before retrying. Consider batching requests.",
rate_limit={"limit": 100, "remaining": 0, "reset_at": reset_timestamp},
))
```
These helpers guarantee `meta.version` is present and prevent ad-hoc response shapes. Avoid constructing dicts manually.
## Implementation Checklist
1. **Import helpers** (`success_response` / `error_response`) in every tool module.
2. **Return `asdict(...)`** so dataclasses serialize with the standardized keys.
3. **Keep `data` payloads business-focused**; put operational context in `meta`.
4. **Document any additional `meta` semantics** (new pagination fields, telemetry) in tool specs.
5. **Record deviations or streaming quirks** in specs to prevent regressions.
## Testing Requirements
- Unit enforcement lives in [`tests/test_responses.py`](../../tests/test_responses.py); extend it when updating helpers.
- Integration tests such as [`tests/integration/test_mcp_tools.py`](../../tests/integration/test_mcp_tools.py) should assert the envelope for new tools.
- Fixtures and parity harnesses must verify `meta.version == "response-v2"` and any declared metadata keys (`warnings`, `pagination`, etc.).
## Compatibility & Rollout Notes
- The `response_contract_v2` feature flag governs client opt-in. During migrations, continue returning v1 only when explicitly required and document timelines in the spec.
- Feature-flag lifecycles must follow [dev_docs/mcp_best_practices/14-feature-flags.md](../mcp_best_practices/14-feature-flags.md), and metadata such as rate limits should align with [dev_docs/mcp_best_practices/02-envelopes-metadata.md](../mcp_best_practices/02-envelopes-metadata.md).
- Telemetry counters in `foundry_mcp/server.py` rely on consistent envelopes; avoid bypassing the helpers or mutating the serialized dict afterward.
## DigestPayload Schema
The `DigestPayload` schema defines the structure for compressed document content in deep research workflows. When a source is digested, its `content` field contains a JSON-serialized DigestPayload.
### Detection
Detect digested sources via the content type:
```python
if source.content_type == "digest/v1":
payload = DigestPayload.from_json(source.content)
```
### DigestPayload Schema (v1.0)
```json
{
"version": "1.0",
"content_type": "digest/v1",
"query_hash": "ab12cd34",
"summary": "Condensed summary of the source content...",
"key_points": [
"First key insight extracted from the document",
"Second key insight with supporting detail"
],
"evidence_snippets": [
{
"text": "Exact quote from the source document...",
"locator": "char:1500-1650",
"relevance_score": 0.85
}
],
"original_chars": 25000,
"digest_chars": 2500,
"compression_ratio": 0.10,
"source_text_hash": "sha256:abc123def456..."
}
```
### DigestPayload Field Definitions
| Field | Type | Required | Constraints | Description |
|-------|------|----------|-------------|-------------|
| `version` | string | YES | Default: `"1.0"` | Schema version identifier |
| `content_type` | string | YES | Default: `"digest/v1"` | Self-describing type for detection |
| `query_hash` | string | YES | Exactly 8 lowercase hex chars, pattern `^[a-f0-9]{8}$` | Hash of the research query for cache keying |
| `summary` | string | YES | Max 2000 chars | Condensed summary of source content |
| `key_points` | array\<string\> | YES | Max 10 items, each max 500 chars | Extracted key insights |
| `evidence_snippets` | array\<EvidenceSnippet\> | YES | Max 10 items | Query-relevant excerpts with locators |
| `original_chars` | int | YES | ≥0 | Character count of original source |
| `digest_chars` | int | YES | ≥0 | Character count of digest output |
| `compression_ratio` | float | YES | 0.0 to 1.0 | Ratio of digest_chars to original_chars |
| `source_text_hash` | string | YES | Pattern `^sha256:[a-f0-9]{64}$` | SHA256 hash of canonical text |
### EvidenceSnippet Schema
```json
{
"text": "Exact substring from the canonical source text...",
"locator": "char:1500-1650",
"relevance_score": 0.85
}
```
| Field | Type | Required | Constraints | Description |
|-------|------|----------|-------------|-------------|
| `text` | string | YES | Max 500 chars | Exact substring from canonical text |
| `locator` | string | YES | See locator formats below | Position reference for citation |
| `relevance_score` | float | YES | 0.0 to 1.0 | Query relevance score |
### Evidence Locator Formats
Locators reference positions in the canonical (normalized) source text:
| Format | Example | Description |
|--------|---------|-------------|
| Text/HTML | `char:1500-1800` | Characters 1500-1799 (exclusive end) |
| PDF | `page:3:char:200-450` | Page 3, characters 200-449 |
**Locator Semantics:**
- Start and end positions are 0-based character indices
- End boundary is exclusive (Python slice convention)
- Page numbers are 1-based (human-readable)
- Offsets reference canonical text (post-normalization)
**Verification:**
```python
# Locators can be verified against archived content
canonical_text[start:end] == snippet.text
```
### Consumer Rules
When processing sources that may contain digests:
1. **Detect** via `source.content_type == "digest/v1"`
2. **Parse** `source.content` as JSON, validate against schema
3. **Skip** further summarization (content is already compressed)
4. **Use** `evidence_snippets` for citations with locators
5. **Use** `digest_chars` for token budget estimation (not `original_chars`)
### Consumer Example
```python
from foundry_mcp.core.research.models import DigestPayload
def process_source(source):
if source.content_type == "digest/v1":
# Parse digest payload
payload = DigestPayload.from_json(source.content)
# Use summary for context (already compressed)
context = payload.summary
# Use key points for highlights
for point in payload.key_points:
print(f"• {point}")
# Use evidence snippets for citations
for ev in payload.evidence_snippets:
print(f'"{ev.text}" [{ev.locator}]')
# Token estimation uses digest size
estimated_tokens = payload.digest_chars // 4
# IMPORTANT: Do NOT re-summarize digested content
return context
else:
# Process raw content normally
return source.content
```
### Serialization Helpers
Use the provided helpers for consistent serialization:
```python
from foundry_mcp.core.research.document_digest import (
serialize_payload,
deserialize_payload,
validate_payload_dict,
)
# Serialize to JSON string
json_str = serialize_payload(payload)
# Deserialize from JSON string
payload = deserialize_payload(json_str)
# Validate dict (e.g., from YAML or manual construction)
payload = validate_payload_dict(data_dict)
```
### Validation Errors
Invalid payloads raise `pydantic.ValidationError` with descriptive messages:
| Error | Cause |
|-------|-------|
| `query_hash: String should match pattern '^[a-f0-9]{8}$'` | Invalid query hash format |
| `summary: String should have at most 2000 characters` | Summary too long |
| `key_points[N]: exceeds maximum length of 500 characters` | Key point too long |
| `relevance_score: Input should be less than or equal to 1` | Score out of range |
| `source_text_hash: String should match pattern '^sha256:[a-f0-9]{64}$'` | Invalid hash format |
### Related Documentation
- Deep Research Guide: [dev_docs/guides/deep-research.md](../guides/deep-research.md)
- Configuration Reference: [dev_docs/configuration.md](../configuration.md)
- Models: [`src/foundry_mcp/core/research/models.py`](../../src/foundry_mcp/core/research/models.py)
## Related References
- Spec: [`response-schema-standardization-2025-11-26-001`](../../specs/completed/response-schema-standardization-2025-11-26-001.json)
- Helpers: [`src/foundry_mcp/core/responses.py`](../../src/foundry_mcp/core/responses.py)
- Testing/fixtures: [dev_docs/mcp_best_practices/10-testing-fixtures.md](../mcp_best_practices/10-testing-fixtures.md)
- Envelopes & metadata guidance: [dev_docs/mcp_best_practices/02-envelopes-metadata.md](../mcp_best_practices/02-envelopes-metadata.md)
