# Aspose MCP Catalog Orchestrator

The Aspose MCP Catalog Orchestrator manages the lifecycle behind `mcp.aspose.com`: discover products, qualify platforms, build versioned manifests, generate landing pages, validate changes, route consequential decisions through human-in-the-loop (HITL) gates, and observe the resulting catalog.

This is more than an MCP content generator. Content generation is one capability within a catalog control plane that separates deterministic catalog data, governance, workflow execution, presentation, and operational reporting.

The current implementation uses only the Python 3.11 standard library and has no package dependencies. Core discovery, qualification, validation, rendering, MCP, and dashboard workflows do not require an LLM, embedding model, API key, Hugo installation, or production hosting. LLM credentials are required only when description optimization is enabled; source-only enrichment remains available.

## What exists today

- Live discovery of the high-code product taxonomy from `products.aspose.com`.
- Deterministic platform qualification with reasons and review markers.
- Versioned raw, qualified, and publication manifests with schemas.
- A generated catalog containing product-family and qualifying platform pages.
- A controlled MCP/CLI proposal interface for product teams.
- Governed extraction of product descriptions, Documentation links, and API Reference links from official Aspose pages.
- Approved agent-oriented MCP descriptions for every product family and canonical platforms for .NET, Java, C++, Python via .NET, Python via Java, Android via Java, Node.js via .NET, Node.js via Java, Node.js via C++, PHP via Java, JavaScript via .NET, JavaScript via C++, Go via C++, Python via C++, and Rust via C++, stored separately from source product descriptions.
- Optional OpenAI-compatible description optimization with source and generation provenance.
- HITL gates for availability, endpoint, repository, public-description, and public-resource changes.
- A local Control Room for approvals, rejections, workflow invocation, health, and guidance.
- Impact previews, one-use confirmation tokens, exact confirmation phrases, and audit events.
- Deterministic validation, anomaly detection, and preservation of approved publication metadata during rediscovery.
- A custom static HTML renderer with packaged templates and structured Product, Documentation, API Reference, and repository cards.

The Control Room is intentionally local-only until company authentication, role-based authorization, protected Git workflow execution, and hosted durable state are designed.

## Content enrichment

Descriptions and resource links are produced through a governed enrichment stage:

```text
product landing page
        |
        v
source description + canonical links
        |
        v
constrained LLM copy optimization
        |
        v
schema and safety validation
        |
        v
HITL proposal in the Control Room
        |
        v
approved landing-page publication
```

The model is an editor, not a factual source. The system preserves the extracted text, canonical source URL, retrieval timestamp, source hash, prompt version, input/output hashes, model, latency, token usage, and validation result. It never stores the API key in manifests, proposals, or audit logs.

Run one controlled target with LLM optimization:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli enrich \
  --product words --platform net \
  --requested-by operator@example.com --limit 1
```

Use `--without-llm` to propose the extracted source description unchanged. Enrichment creates pending proposals; it does not mutate the public manifest or generated pages. Resource candidates are limited to HTTPS URLs on Aspose domains and verified before they are proposed; unavailable links remain explicitly marked `not_discovered`. The Control Room shows source and proposed content, Documentation/API Reference links, gate reasons, and the exact publication effect before approval.

Copy `.env.example` to `.env` and set `API_KEY`, `LLM_URL`, and `LLM_MODEL`. `LLM_URL` is the HTTPS OpenAI-compatible API base URL; the client calls its `/chat/completions` endpoint and requests a JSON object. `.env` and `.env.*` are Git-ignored, while `.env.example` is explicitly safe to commit. Credentials are read locally and are not written to proposals, manifests, generated pages, or audit logs.

## API Reference discovery

API Reference links are resolved independently from LLM enrichment. The resolver first verifies the canonical `reference.aspose.com/{family}/{platform}/` route, then tries an explicitly mapped underlying runtime for wrapper SDKs, and finally uses a clearly labelled family-level fallback. Aspose.Total is intentionally mapped to the API Reference root because `/total/` is not a valid reference route.

No URL is published solely because it matches a naming convention. Verification requires HTTPS, the exact `reference.aspose.com` host, a successful response, and a destination path consistent with the candidate. Temporary failures are retried. An available reference is never downgraded to `not_discovered` by a later content-enrichment or inventory-refresh run.

Preview the current catalog-wide resolution without changing the manifest:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli refresh-api-references
```

An operator may apply the freshly verified snapshot explicitly:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli refresh-api-references \
  --apply --approved-by reviewer@example.com
```

The preferred interactive path is the Control Room's **Refresh API references** workflow. Its network verification happens during the impact preview; confirmation applies that exact in-memory snapshot without refetching. The preview reports changed, fallback, and unresolved targets before the human types `REFRESH API REFERENCES`.

## Documentation discovery

Documentation links use a separate resolver with the same verification and review guarantees. It checks `docs.aspose.com/{family}/{platform}/` first, then a clearly labelled underlying-runtime route for wrapper SDKs, and finally the family documentation. Only HTTPS links on the exact `docs.aspose.com` host with a matching destination path are published.

Documentation can be refreshed across the complete catalog, or limited to selected product families. Preview and apply the exact reviewed catalog-wide snapshot with:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli refresh-documentation
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli refresh-documentation \
  --apply --approved-by reviewer@example.com
```

Use `--family total` or another family slug to limit a CLI refresh. The Control Room exposes the catalog-wide operation as **Refresh documentation** and requires the exact confirmation phrase `REFRESH DOCUMENTATION`. If an exact platform or wrapper route does not exist, its card states the runtime or family scope instead of presenting the fallback as platform-specific documentation.

## Architecture

```text
products.aspose.com
        |
        v
data/products.raw.json             discovery evidence
        |
        v
data/products.qualified.json       deterministic decisions and reasons
        |
        v
data/mcp-site-manifest.json        public catalog source of truth
        |                         |
        v                         v
presentation renderer      MCP/CLI proposals
        |                         |
        v                         v
public/ landing pages      Control Room + HITL
        ^                         |
        |                         v
        +---------- approved metadata
```

The modules under `src/aspose_mcp_catalog` keep discovery, qualification, source-content extraction, LLM transport, enrichment, transformation, rendering, updates, anomaly detection, audit logging, orchestration, CLI transport, MCP transport, and the dashboard action layer separate. Content data remains independent of the current page template so future renderers can consume the same approved manifest.

## Product and component names

- **Aspose MCP Catalog Orchestrator** - the complete lifecycle system.
- **Catalog Builder** - discovery, qualification, validation, and manifest generation.
- **Layout Renderer** - presentation of the normalized site manifest.
- **MCP Control Room** - HITL decisions, workflow invocation, catalog health, and operator guidance.
- **Catalog Steward** - governance responsibilities such as approval, rejection, and auditability.

## Run locally

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli pipeline
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli check
python3 -m http.server --directory public 8080
```

## Deploy to GitHub Pages

The workflow in `.github/workflows/deploy-pages.yml` validates the catalog,
runs the tests, builds the `kb` Hugo preset from the checked-in manifest, and
deploys it to GitHub Pages on every push to `main`. It can also be started
manually from the repository's **Actions** tab.

For the first deployment, open **Settings > Pages** in GitHub and set
**Source** to **GitHub Actions**. Push the workflow to `main` (or run it
manually), then follow the deployment URL shown by the `deploy` job. For this
project repository, the default URL is:

```text
https://tahir-manzoor.github.io/aspose-mcp-catalog-orchestrator/
```

The workflow obtains the Pages base URL from GitHub, so generated CSS links and
catalog routes work both at the repository URL above and if a custom domain is
configured later.

For a reproducible or offline run, save the source page and pass `--html-file`:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli pipeline --html-file products.html
```

Runtime audit logs are written to `.runtime/logs/catalog.jsonl`; that directory is intentionally ignored by Git. Events include a run ID, stage, action, result, and relevant entity context. Sensitive key names are redacted.

## Qualification and HITL

The initial approved policy excludes SharePoint, Reporting Services, and JasperReports. Rules live in `config/qualification.json`; every platform remains visible in the qualified manifest with its decision and reason.

Human approval is required when a proposal:

- changes an entry to `available`;
- changes an endpoint;
- changes a repository;
- changes a public description;
- changes public resource links.

An available entry must declare tools, endpoints, and a repository. Content-enrichment proposals carry their extracted source, canonical URL, hashes, and generation metadata for review. The optional `mcp_description` keeps agent-facing MCP copy separate from the source product description and follows the same public-description HITL gate. A pending description proposal can be revised without overwriting its evidence: the original becomes `superseded`, the edited text is placed in a linked replacement proposal, and that replacement still requires an independent HITL approval. Public HTML is never edited directly; it is regenerated from the approved site manifest.

## Product-team update workflow

Create a proposal:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli propose-update \
  --product pdf --platform net \
  --mcp-description "Agent-facing MCP description of at least 20 words" \
  --requested-by product-team@example.com
```

After review, apply it and regenerate the site:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli apply-update \
  changes/pending/PROPOSAL.json --approved-by reviewer@example.com
PYTHONPATH=src python3 -m aspose_mcp_catalog.cli generate
```

The CLI keeps approval and generation as separate operator steps. Approving through the Control Room performs both operations and reports the regenerated page count.

The stdio MCP server exposes `update_product_mcp`, which validates and queues the same proposal without mutating the live catalog:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.mcp_server
```

Configure that command in an MCP host with this repository as its working directory. Applying approved proposals deliberately remains an operator/CI action.

## MCP Control Room

The Control Room provides a frontend for HITL review, safe workflow invocation, catalog health, and an operator playbook:

```bash
PYTHONPATH=src python3 -m aspose_mcp_catalog.dashboard
```

Open `http://127.0.0.1:8765/dashboard/`. The server intentionally accepts only loopback addresses until production authentication is designed.

Every consequential action follows the same contract:

1. The first button requests a server-generated impact preview.
2. The preview lists affected fields, files, routes, HITL gates, and network activity.
3. The operator supplies an identity and types the exact displayed confirmation phrase.
4. The server accepts the preview token once, within five minutes, then performs the action and writes an audit event.

The Control Room can revise, approve, or reject pending product-description and `mcp_description` proposals; invoke one-target content enrichment with or without LLM optimization; refresh verified API Reference and documentation resources across the catalog; validate the catalog; regenerate public pages; and run the full live inventory refresh. Proposal cards and approval previews show the exact current and proposed MCP wording. The revision editor preserves the other proposed fields and provenance, validates the replacement description, and previews the exact before-and-after wording before requiring `REVISE <proposal-id>`. Confirmation supersedes the original and creates a linked pending proposal; it does not publish the revision, which requires a separate approval. Enrichment execution likewise only queues a proposal for HITL review. Resource refreshes apply only the exact snapshot shown in their impact preview. Rejections require a reason and never alter public catalog metadata. Approvals update the manifest and regenerate the site. Dashboard logs are written beneath `.runtime/logs/` and remain local.

Inventory refresh uses a true dry run: the preview fetches and validates the live taxonomy without writing, displays route and count differences, and stores that snapshot in the short-lived preview. Confirmation applies the exact reviewed snapshot rather than fetching potentially different data.

This local dashboard is an operational prototype, not a production authentication boundary. Before remote deployment, add company identity/SSO, role-based authorization, protected GitLab merge-request execution, and hosted durable storage.

## Presentation renderers

The catalog lifecycle is presentation-agnostic. `site.py` remains the operational renderer, while the opt-in Hugo adapter provides a reviewable `kb` preset without changing the manifest or committed `public/` output. See [docs/hugo-renderer.md](docs/hugo-renderer.md) for its boundary, build commands, configuration, security notes, and acceptance checklist.

The intended presentation architecture is:

```text
canonical site manifest
          |
          v
normalized page model
          |
          +-- StaticHtmlRenderer (current)
          +-- HugoRenderer (kb preview)
          +-- future renderer adapters
                    |
                    v
              layout preset
```

The layout module provides a stable renderer interface, an explicit preset registry, and small theme-specific adapters. A preset defines how the canonical catalog maps to Hugo content descriptors, data, partials, and configuration. Approved copy and catalog fields remain in the manifest rather than being duplicated in theme files.

The goal is a curated preset list rather than claiming compatibility with every arbitrary Hugo theme. Hugo themes differ in their expected content structure and template contracts, so each supported preset must be explicitly adapted and validated.

The Control Room theme-selection workflow is expected to:

1. Select a renderer and supported preset.
2. Display capabilities, compatibility, and configuration options.
3. Generate a non-publishing preview.
4. Show affected files, routes, and presentation changes.
5. Require an explicit human confirmation.
6. Save the reviewed selection and regenerate the site.

Changing the layout will be HITL-controlled because it can affect every public page even when product availability metadata remains unchanged.

## Verification

```bash
PYTHONPATH=src python3 -m unittest discover -s tests -v
```

The tests cover HTML discovery and slug normalization, qualification exclusions, generated routes, canonical API Reference and documentation routes, runtime and family fallbacks, cross-family isolation, resource preservation, source description/resource extraction, structured LLM responses, credential redaction, anomaly checks, approval enforcement, enrichment gates, immutable product and MCP-description revisions and lineage, dashboard preview tokens, exact confirmation, rejection reasons, and proposal-only publication safety.

## Repository layout

- `config/` - deterministic policy configuration.
- `schemas/` - versioned contracts for each lifecycle boundary.
- `data/` - current raw, qualified, and public manifests.
- `changes/pending/` - structured product-team proposals.
- `reports/` - deterministic anomaly reports.
- `public/` - generated static landing pages.
- `.env.example` - safe local LLM configuration template; the real `.env` remains ignored.
- `src/aspose_mcp_catalog/api_references.py` - verified API Reference candidates, fallbacks, and catalog refresh.
- `src/aspose_mcp_catalog/documentation.py` - verified documentation candidates, fallbacks, and selective family refresh.
- `src/aspose_mcp_catalog/platforms.py` - shared wrapper-to-runtime platform relationships.
- `src/aspose_mcp_catalog/resources.py` - no-downgrade resource merging.
- `src/aspose_mcp_catalog/url_verification.py` - strict official-host and destination verification.
- `src/aspose_mcp_catalog/content.py` - official-page description and resource extraction.
- `src/aspose_mcp_catalog/descriptions.py` - shared public-description normalization and validation.
- `src/aspose_mcp_catalog/llm.py` - OpenAI-compatible structured completion transport.
- `src/aspose_mcp_catalog/enrichment.py` - governed enrichment and proposal creation.
- `src/aspose_mcp_catalog/dashboard_ui/` - responsive Control Room frontend.
- `src/aspose_mcp_catalog/` - remaining reusable lifecycle, transport, rendering, and dashboard modules.
- `tests/` - dependency-free automated tests and fixtures.

## Next decisions

Before production deployment, the product families in the checked-in `products.qualified.json` matrix will be divided between @tahirmanzoor and @imranwar. Each reviewer will work independently on only their assigned product families, verify every product/platform qualification and its recorded reason, and record approval for their assigned subset. A subset will be treated as production-approved only after its assigned reviewer signs off. Material changes to a product family's qualification or the applicable policy should return to the reviewer responsible for that family.

I and @fahadaspose will work on the next phases, including company authentication, GitLab workflow integration, the pluggable renderer contract, curated Hugo presets, CI/CD, DNS, hosting, and the global Aspose Support-menu link.