# TOON Encoder Plugin
> **Document Location:** `plugins/toon_encoder/README.md`
>
> **Related Documents:**
> - [TOON Format Specification](https://github.com/toon-format/spec/blob/main/SPEC.md)
> - Tests: `tests/unit/plugins/toon_encoder/`
A post-invoke plugin that converts JSON tool results to [TOON (Token-Oriented Object Notation)](https://github.com/toon-format/spec) format for reduced token consumption when sending responses to LLM agents.
## Overview
TOON is a compact, human-readable encoding format designed specifically for LLM prompts. It provides lossless serialization of JSON data in a syntax that minimizes tokens by:
- Eliminating quotation marks around keys and simple string values
- Using compact array syntax: `field[N]: val1,val2,val3`
- Using columnar format for homogeneous object arrays
- Removing unnecessary punctuation where context is clear
## Token Savings
### Expected Savings by Data Type
| Data Type | JSON Size | TOON Size | Savings |
|-----------|-----------|-----------|---------|
| Simple object | 150 bytes | 80 bytes | **~47%** |
| Array of 10 objects | 2 KB | 600 bytes | **~70%** |
| Nested data | 5 KB | 2 KB | **~60%** |
| Simple primitives | minimal | minimal | ~0% |
### Verified Example
Real-world test with array of user objects:
```
JSON (231 chars):
{"users": [{"id": 0, "name": "user0", "active": true}, {"id": 1, "name": "user1", "active": true}, ...]}
TOON (109 chars):
users:
[5]{active,id,name}:
true,0,user0
true,1,user1
true,2,user2
true,3,user3
true,4,user4
Savings: 52.8% (122 bytes saved)
```
### Why This Matters
For an agent using MCP Gateway to federate 10+ MCP servers:
- Each tool call returns JSON responses (database queries, API data, etc.)
- JSON verbosity (quoted keys, quoted strings, punctuation) inflates token count
- **Accumulated impact across many tool calls:**
- Inference latency (more tokens = longer processing)
- Context window consumption (less room for reasoning)
- API costs (for cloud-hosted inference)
## Running Locally
### Prerequisites
- Python 3.11+
- MCP Gateway repository cloned
- `uv` package manager (or pip)
### Step 1: Set Up Development Environment
```bash
# Clone the repo (if not already)
git clone https://github.com/IBM/mcp-context-forge.git
cd mcp-context-forge
# Create virtual environment and install dependencies
make venv install-dev
# Copy environment template
cp .env.example .env
```
### Step 2: Enable the Plugin
Edit `.env` to enable plugins:
```bash
PLUGINS_ENABLED=true
PLUGIN_CONFIG_FILE=plugins/config.yaml
```
Add the TOON encoder to `plugins/config.yaml`:
```yaml
plugins:
# ... existing plugins ...
- name: "ToonEncoder"
kind: "plugins.toon_encoder.toon_encoder.ToonEncoderPlugin"
hooks: ["tool_post_invoke"]
mode: "enforce"
priority: 900
config:
min_size_bytes: 100
```
### Step 3: Start the Gateway
```bash
# Development server with auto-reload
make dev
# Or production server
make serve
```
The gateway will start on `http://localhost:8000` (dev) or `http://localhost:4444` (prod).
### Step 4: Verify Plugin is Loaded
Check the startup logs for:
```
INFO: ToonEncoderPlugin initialized: min_size=100, max_size=1048576, exclude=[]
```
### Step 5: Test the Plugin
```bash
# Run plugin-specific tests
pytest tests/unit/plugins/toon_encoder/ -v
# Quick inline test of TOON encoder
python3 -c "
from plugins.toon_encoder.toon import encode, decode
data = {'users': [{'id': 1, 'name': 'alice'}, {'id': 2, 'name': 'bob'}]}
toon = encode(data)
print('TOON output:')
print(toon)
print()
print('Round-trip:', decode(toon) == data)
"
```
### Step 6: Test with Real Tool Calls
1. Register an MCP server via the Admin UI or API
2. Create a virtual server with tools
3. Invoke a tool that returns JSON
4. Check the response for `annotations.format: "toon"`
```bash
# Example: Call a tool via the API
curl -X POST http://localhost:8000/tools/my-tool/invoke \
-H "Content-Type: application/json" \
-d '{"arguments": {}}'
# Response should contain TOON-encoded content with format annotation
```
### Troubleshooting
| Issue | Solution |
|-------|----------|
| Plugin not loading | Check `PLUGINS_ENABLED=true` in `.env` |
| No TOON conversion | Verify JSON response > `min_size_bytes` (default 100) |
| Import errors | Run `make install-dev` to install dependencies |
| Tests failing | Ensure you're in the virtual environment |
---
## Installation
The plugin is included in the MCP Gateway plugins directory. Enable it in your configuration:
### 1. Add to `plugins/config.yaml`
```yaml
plugins:
- name: "ToonEncoder"
kind: "plugins.toon_encoder.toon_encoder.ToonEncoderPlugin"
hooks: ["tool_post_invoke"]
mode: "enforce"
priority: 900 # Run late, after other post-processing
config:
min_size_bytes: 100 # Only convert JSON larger than this
max_size_bytes: 1048576 # Skip very large payloads (1MB)
exclude_tools: [] # Tool names to skip
include_tools: null # Whitelist (null = all tools)
add_format_marker: true # Add annotation marking content as TOON
skip_on_error: true # Continue with JSON if conversion fails
```
### 2. Enable plugins in `.env`
```bash
PLUGINS_ENABLED=true
PLUGIN_CONFIG_FILE=plugins/config.yaml
```
## Configuration Options
| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `min_size_bytes` | int | 100 | Minimum JSON size to trigger conversion |
| `max_size_bytes` | int | 1048576 | Maximum JSON size to convert (prevents OOM) |
| `exclude_tools` | list | [] | Tool names to skip conversion |
| `include_tools` | list | null | Whitelist of tools (if set, only these are converted) |
| `add_format_marker` | bool | true | Add `annotations.format: "toon"` to converted content |
| `skip_on_error` | bool | true | Continue with original JSON if conversion fails |
## Usage Examples
### Example 1: Convert all tool results
```yaml
- name: "ToonEncoder"
kind: "plugins.toon_encoder.toon_encoder.ToonEncoderPlugin"
hooks: ["tool_post_invoke"]
mode: "enforce"
priority: 900
config:
min_size_bytes: 100
```
### Example 2: Only convert specific tools
```yaml
- name: "ToonEncoder"
kind: "plugins.toon_encoder.toon_encoder.ToonEncoderPlugin"
hooks: ["tool_post_invoke"]
mode: "enforce"
priority: 900
config:
include_tools:
- "search_documents"
- "query_database"
- "list_items"
```
### Example 3: Exclude binary/streaming tools
```yaml
- name: "ToonEncoder"
kind: "plugins.toon_encoder.toon_encoder.ToonEncoderPlugin"
hooks: ["tool_post_invoke"]
mode: "enforce"
priority: 900
config:
exclude_tools:
- "download_file"
- "stream_data"
- "get_binary"
```
## TOON Format Reference
TOON (Token-Oriented Object Notation) is designed to minimize tokens while preserving all JSON data.
For the complete specification, see the [official TOON spec](https://github.com/toon-format/spec/blob/main/SPEC.md).
### Primitives
| JSON | TOON | Notes |
|------|------|-------|
| `null` | `null` | Literal |
| `true` | `true` | Literal |
| `false` | `false` | Literal |
| `42` | `42` | Integers as-is |
| `3.14` | `3.14` | Floats as-is |
| `"hello"` | `hello` | Simple strings unquoted |
| `"hello world"` | `hello world` | Spaces OK in values |
| `"has,comma"` | `"has,comma"` | Special chars quoted |
| `""` | `""` | Empty string quoted |
### Objects
**Simple Object:**
```
JSON: {"name": "alice", "age": 30, "active": true}
TOON:
name: alice
age: 30
active: true
```
**Nested Object:**
```
JSON: {"user": {"name": "alice", "email": "alice@example.com"}}
TOON:
user:
name: alice
email: alice@example.com
```
**Empty Object:**
```
JSON: {}
TOON: (empty document)
```
### Arrays
**Simple Array (inline):**
```
JSON: [1, 2, 3]
TOON: [3]: 1,2,3
```
**String Array:**
```
JSON: ["apple", "banana", "cherry"]
TOON: [3]: apple,banana,cherry
```
**Empty Array:**
```
JSON: []
TOON: [0]:
```
**Nested Array:**
```
JSON: {"tags": ["python", "fastapi", "mcp"]}
TOON:
tags[3]: python,fastapi,mcp
```
### Columnar Format (Arrays of Objects)
When all objects in an array have identical keys with primitive values, TOON uses
a compact columnar format:
```
JSON:
[
{"id": 1, "name": "alice", "score": 95},
{"id": 2, "name": "bob", "score": 87},
{"id": 3, "name": "charlie", "score": 92}
]
TOON:
[3]{id,name,score}:
1,alice,95
2,bob,87
3,charlie,92
```
The header `[3]{id,name,score}:` specifies:
- `[3]` - Array contains 3 objects
- `{id,name,score}` - Column order (alphabetically sorted)
- Each row contains values in that column order
### Complex Nested Example
```
JSON:
{
"status": "success",
"data": {
"users": [
{"id": 1, "name": "alice", "roles": ["admin", "user"]},
{"id": 2, "name": "bob", "roles": ["user"]}
],
"total": 2
},
"metadata": {
"page": 1,
"per_page": 10
}
}
TOON:
status: success
data:
users:
[2]:
id: 1
name: alice
roles:
[2]: admin,user
id: 2
name: bob
roles:
[1]: user
total: 2
metadata:
page: 1
per_page: 10
```
### String Quoting Rules
Strings must be quoted if they:
- Are empty
- Equal `true`, `false`, or `null`
- Look like a number (e.g., `"123"`, `"3.14"`)
- Contain special characters: `,` `:` `[` `]` `{` `}` `"` `\` newlines
- Have leading or trailing whitespace
```
JSON: {"status": "null", "count": "42", "note": "a,b,c"}
TOON:
status: "null"
count: "42"
note: "a,b,c"
```
### Key Quoting Rules
Object keys must be quoted if they don't match `^[A-Za-z_][A-Za-z0-9_.]*$`:
```
JSON: {"valid_key": 1, "has space": 2, "123numeric": 3}
TOON:
valid_key: 1
"has space": 2
"123numeric": 3
```
## Agent Integration (Required)
> **IMPORTANT:** When you enable the TOON encoder plugin, your agent's system prompt **must be updated** to understand TOON format. Without this change, your agent will not be able to parse tool results correctly.
### Why This Is Required
- The gateway converts JSON tool responses to TOON format before sending to your agent
- Standard LLMs are trained on JSON, not TOON
- Your agent needs instructions on how to interpret TOON-formatted data
- Tool results will include `annotations.format: "toon"` to indicate the format
### Recommended System Prompt Addition
Add the following to your agent's system prompt:
```
## Tool Response Format: TOON
Tool results from this gateway use TOON (Token-Oriented Object Notation) format for efficiency.
Results with `annotations.format: "toon"` are TOON-encoded. Parse them as follows:
### TOON Syntax Rules
1. **Objects** - Key-value pairs on separate lines (no braces, no quoted keys):
```
name: alice
age: 30
active: true
```
Equivalent JSON: {"name": "alice", "age": 30, "active": true}
2. **Simple Arrays** - Count prefix with comma-separated values:
```
[3]: apple,banana,cherry
```
Equivalent JSON: ["apple", "banana", "cherry"]
3. **Object Arrays (Columnar)** - Count and keys in header, values in rows:
```
[3]{id,name,score}:
1,alice,95
2,bob,87
3,charlie,92
```
Equivalent JSON: [{"id": 1, "name": "alice", "score": 95}, {"id": 2, "name": "bob", "score": 87}, {"id": 3, "name": "charlie", "score": 92}]
4. **Nested Objects** - Indented under parent key:
```
user:
name: alice
email: alice@example.com
settings:
theme: dark
notifications: true
```
5. **Special Values**:
- `null` = null/None
- `true` / `false` = booleans
- Unquoted numbers = integers or floats
- Quoted strings `"value"` = strings containing special characters (commas, colons, newlines)
### Parsing Instructions
When you receive a tool result:
1. Check if `annotations.format` equals `"toon"`
2. If yes, parse the text content using TOON rules above
3. If no annotation or format is not "toon", parse as standard JSON
TOON is a lossless format - all data from the original JSON is preserved.
```
### Complete System Prompt Example
Here's a complete example for an agent using MCP Gateway with TOON:
```
You are a helpful assistant with access to tools via MCP Gateway.
## Tool Response Format: TOON
Tool results from this gateway use TOON (Token-Oriented Object Notation) format for efficiency.
TOON reduces token usage by 40-70% compared to JSON while preserving all data.
### Quick Reference
| TOON | Meaning |
|------|---------|
| `key: value` | Object property |
| `[N]: a,b,c` | Array of N items |
| `[N]{k1,k2}:` followed by rows | Array of N objects with keys k1,k2 |
| `null`, `true`, `false` | Literal values |
| `"quoted"` | String with special characters |
### Example
TOON:
```
users:
[2]{id,name,active}:
1,alice,true
2,bob,false
total: 2
```
Means:
```json
{
"users": [
{"id": 1, "name": "alice", "active": true},
{"id": 2, "name": "bob", "active": false}
],
"total": 2
}
```
When processing tool results, interpret TOON format and extract the data you need.
```
### Checking Format Annotation
Tool results include a format marker when TOON-encoded:
```python
# Result structure from gateway
{
"content": [
{
"type": "text",
"text": "name: alice\nage: 30",
"annotations": {
"format": "toon" # <-- This indicates TOON encoding
}
}
]
}
```
## Programmatic Usage
The TOON encoder/decoder can be used directly:
```python
from plugins.toon_encoder.toon import encode, decode, estimate_token_savings
# Encode Python object to TOON
data = {"users": [{"id": 1, "name": "alice"}, {"id": 2, "name": "bob"}]}
toon_str = encode(data)
print(toon_str)
# users:
# [2]{id,name}:
# 1,alice
# 2,bob
# Decode TOON back to Python
decoded = decode(toon_str)
assert decoded == data
# Estimate savings
import json
json_str = json.dumps(data)
json_len, toon_len, savings_pct = estimate_token_savings(json_str)
print(f"Saved {savings_pct:.1f}% ({json_len - toon_len} bytes)")
```
## Monitoring
The plugin tracks conversion statistics accessible via the `get_stats()` method:
```python
stats = plugin.get_stats()
# {
# "conversions_attempted": 150,
# "conversions_successful": 142,
# "total_bytes_saved": 45230,
# "success_rate": 94.67
# }
```
Conversion metadata is also included in each response:
```python
result.metadata = {
"toon_encoded": True,
"bytes_saved": 1250,
"savings_percent": 62.5,
"conversion_time_ms": 0.45
}
```
## Testing
Run the plugin tests:
```bash
# Run TOON encoder tests
pytest tests/unit/plugins/toon_encoder/test_toon.py -v
# Run plugin integration tests
pytest tests/unit/plugins/toon_encoder/test_toon_encoder.py -v
# Run all plugin tests
pytest tests/unit/plugins/toon_encoder/ -v
```
## Limitations
1. **Agent must understand TOON** - The receiving LLM/agent needs to parse TOON format
2. **Not all data benefits** - Very simple data may not see significant savings
3. **No streaming support** - Entire response must be buffered for conversion
### Known Decoder Limitations
The built-in TOON decoder has some edge case limitations (these don't affect the plugin's
primary use case of encoding for LLMs, which parse the format themselves):
- **Quoted keys in single-line objects** - Objects with quoted keys like `"has space": 1`
are encoded correctly but may not decode properly when on a single line
- **Arrays of arrays** - Nested arrays like `[[1,2], [3,4]]` may have decoding issues
- **Mixed sibling fields after columnar arrays** - When a columnar array is followed by
primitive fields at the same nesting level, the primitives may not decode
These limitations affect the `decode()` function used for round-trip testing. The `encode()`
function correctly handles all these cases per the TOON specification.
## License
Apache-2.0 - See LICENSE file in repository root.
