# PipSync MCP Evals

Public safety evaluations and a no-execution MCP mock for trading-oriented tool designs.

The repository models three boundaries:

- read-only synthetic signal listing;
- a deterministic, non-executable paper preview;
- a live-capable confirmation **schema** that the public mock always blocks.

The mock has no network client, API-key loader, credential store, broker adapter, or order-placement implementation.

## Run the evaluations

```bash
PYTHONPATH=src python -m pipsync_mcp_evals.evals
PYTHONPATH=src python -m unittest discover -s tests -v
```

## Run the stdio mock

```bash
PYTHONPATH=src python -m pipsync_mcp_evals.server
```

It accepts one JSON-RPC message per line and implements the modern, stateless MCP
2026-07-28 surface: `server/discover`, `tools/list`, and `tools/call`. Every request
must carry `io.modelcontextprotocol/protocolVersion` and
`io.modelcontextprotocol/clientCapabilities` in `params._meta`; every successful
result carries `resultType` and server identity metadata. Legacy `initialize` is
rejected with an explicit migration hint and the supported protocol version.

## What the evaluations enforce

- Every tool has explicit MCP annotations.
- Tool input schemas reject unknown fields.
- Confirmation is classified as non-read-only and destructive.
- Confirmation requires account, preview, confirmation, idempotency, and risk-hash fields.
- Missing confirmation data fails closed.
- Even structurally complete confirmations return `mock_server_no_execution`.
- Preview output is deterministic and explicitly non-executable.
- Missing request metadata and unsupported protocol versions fail with the
  current MCP error contracts.

Annotations are hints, not an authorization boundary. Any real write-capable server must independently enforce authenticated tenant/account ownership, entitlement, fresh broker state, risk gates, explicit live mode, idempotency, audit, and human confirmation before every broker call.

The catalog targets the [MCP 2026-07-28 tools contract](https://modelcontextprotocol.io/specification/2026-07-28/server/tools)
and its [stateless discovery contract](https://modelcontextprotocol.io/specification/2026-07-28/server/discover).

## Continue with PipSync

This repository deliberately cannot execute. For the managed connector setup,
capability boundaries, and confirmation flow, use the
[PipSync MCP quickstart](https://pipsync.io/en/mcp-quickstart-guide?utm_source=github&utm_medium=repository&utm_campaign=developer-tools&utm_content=pipsync-mcp-evals).

## License

Apache-2.0.