AEO Graph Engine (aeo-graph-engine)

Autonomous Answer Engine Optimization (AEO/GEO), Live Site Crawler, Schema.org Linked Data @graph, llms.txt & AEO Studio UI Workbench

🌐 Overview

Traditional SEO was engineered for classic 10-blue-link search result pages. Answer Engine Optimization (AEO) and Generative Engine Optimization (GEO) optimize your website for direct conversational retrieval and factual synthesis by ChatGPT Search, Perplexity AI, Claude, Google AI Overviews, Apple Intelligence, and Grok.

aeo-graph-engine is a zero-runtime-dependency Python engine, live multi-page crawler, CLI, and Interactive Web Workbench (design influenced by Material 3) extracted and modularized from the Zoth Studio architecture. It audits live URLs, generates interconnected Schema.org @graph JSON-LD, compiles standard llms.txt manifests, and optimizes AI bot directives.

                  ┌──────────────────────────────────────────────────────────┐
                  │                 YOUR APPLICATION / SITE                 │
                  └────────────────────────────┬─────────────────────────────┘
                                               │
                                 aeo scan https://mysite.com
                                               │
                ┌──────────────────────────────┼───────────────────────────────┐
                ▼                              ▼                               ▼
   ┌─────────────────────────┐    ┌───────────────────────────┐   ┌──────────────────────────┐
   │   schema-graph.json     │    │   llms.txt & llms-full    │   │    ai.txt & robots.txt   │
   │ • Organization          │    │ • llmstxt.org compliant   │   │ • Bot Access Policies    │
   │ • WebSite SearchAction  │    │ • Summary blockquotes     │   │ • GPTBot / Perplexity    │
   │ • SoftwareApp / Service │    │ • Deep research ontology  │   │ • ClaudeBot / Applebot   │
   │ • FAQPage (Q&A citations)│   │ • Direct markdown links   │   │ • Google-Extended rules  │
   └─────────────────────────┘    └───────────────────────────┘   └──────────────────────────┘

Related MCP server: crawlie-mcp

🎨 AEO Studio Workbench (Design Influenced by Material 3)

aeo-graph-engine includes a built-in interactive Light Mode Web UI (AEO Studio) with design influenced by Material 3: clean #f8f9fa canvas, elevation cards, live entity graph previews, and a real-time 0–100 AEO readiness meter.

Launch the Local Studio:

# Launch interactive local studio (runs on http://127.0.0.1:8080 or specified port)
aeo serve --port 8080

# Or open public/index.html directly in any browser (100% offline static app)
open public/index.html

Studio Workbench Capabilities:

• 🌐 Live Website Scanner & Multi-Page Crawler: Enter any live URL (or localhost) to audit real sitemaps, robots.txt, subpages, and calculate live AEO scores.

• ⚡ AI Keys & Local Ports Gateway: Configure API keys for OpenAI, Claude, Groq, Perplexity or connect free local models (Ollama, LM Studio, vLLM) on custom ports.

• 🪄 AI Prompt Synthesizer: Type a 1-sentence prompt (e.g. "DeFi lending on Solana called SolarYield") to auto-generate full Schema, llms.txt, and metadata.

• 🤖 AI Agent & MCP Hub: Direct copy-paste configs for Claude Desktop, Cursor, Cline, Zed, and agent system prompt templates.

• 📦 Framework Exporter: One-click production code for Next.js App Router, Astro, Vite + React, SvelteKit, Remix, Nuxt, and Hugo/Jekyll.

• 📊 Real-time 0–100 AEO Readiness Meter: Real weighted scores across 5 core dimensions (Schema, llms.txt, AI bots, content, technical SEO).

• 🕸️ Visual Entity Graph & Schema Preview: Live syntax-highlighted @graph JSON-LD generator with canonical @id linking.

• 📄 llms.txt & llms-full.txt Compiler: llmstxt.org-compliant machine index with structured markdown links.

• 🤖 AI Bot Access Matrix: Live compatibility breakdown for GPTBot, PerplexityBot, ClaudeBot, Applebot, and Google-Extended.

• 💉 Live HTML Schema Injector: Paste raw HTML to test idempotent script embedding.

• 🏛️ AEO Backlink & Citation Playbook: Actionable high-authority citation targets and niche distribution strategy.

• 📥 One-Click .zip Export: Download all generated files in a single clean archive.

⚡ AI Agent API Keys & Local Endpoints / Port Gateway

AEO Studio includes a built-in AI Provider & Custom Port Management Gateway accessible directly from the UI or via REST API:

• 🔑 Cloud AI Provider API Keys:

  • OpenAI (OPENAI_API_KEY): GPT-4o, GPT-4o-mini, o1, o3-mini

  • Anthropic (ANTHROPIC_API_KEY): Claude 3.7 Sonnet, Claude 3.5 Sonnet, Claude 3.5 Haiku

  • Perplexity AI (PERPLEXITY_API_KEY): Sonar Pro, Sonar Reasoning

  • Groq (GROQ_API_KEY): Llama-3.3 70B, DeepSeek R1 Distill (ultra low-latency)

  • OpenRouter (OPENROUTER_API_KEY): Access 200+ frontier models with one key

  • DeepSeek (DEEPSEEK_API_KEY): DeepSeek V3 & R1

• 🔌 Local LLM Endpoints & Ports (100% Free & Private):

  • Ollama: Default http://127.0.0.1:11434 (Port 11434)

  • LM Studio: Default http://127.0.0.1:1234/v1 (Port 1234)

  • vLLM / llama.cpp: Default http://127.0.0.1:8000/v1 (Port 8000)

  • Hermes Agent / Zoth Memory Daemon: Default http://127.0.0.1:8788/v1 (Port 8788)

  • AEO MCP Gateway Port: Port 8091

• ⚡ Live Endpoint Latency & Connection Tester: Ping any endpoint with 1 click to verify available models and response times.

• 🔮 Live Answer Engine Perception Simulator: Query simulated ChatGPT Search & Perplexity engines to verify how AI answer engines perceive and cite your domain.

🤖 Model Context Protocol (MCP) Server & AI Agent Hub

aeo-graph-engine exposes a zero-dependency JSON-RPC 2.0 Model Context Protocol (MCP) server over stdio. Connect any AI coding agent (Claude Desktop, Cursor, Cline, Zed, Hermes, OpenCode, AutoGen, CrewAI) so they can autonomously crawl URLs, generate Schema @graphs, and optimize your codebase.

👉 Full MCP Setup Guide & Documentation: docs/MCP_GUIDE.md
👉 Agent Workflow & Autonomous Optimization: docs/AI_AGENT_INTEGRATION.md

Connect with Claude Desktop / Cursor in 30 Seconds:

Run the config generator to get your exact configuration block:

aeo mcp --config claude_desktop
# or
aeo mcp --config cursor

Add to Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "aeo-graph-engine": {
      "command": "python3",
      "args": ["-m", "aeo_graph_engine.mcp_server"]
    }
  }
}

6 Native MCP Tools Available to AI Agents:

1. aeo_scan_site: Live URL crawler, multi-page audit, 0-100 AEO score, bot permissions.

2. aeo_synthesize_prompt: Turn natural language descriptions into complete structured AEO configs.

3. aeo_generate_bundle: Generate schema-graph.json, llms.txt, ai.txt, and robots.txt.

4. aeo_inject_html: Embed Schema.org JSON-LD directly into HTML files.

5. aeo_validate: Diagnostic audit and 0-100 readiness verification.

6. aeo_get_framework_snippets: Retrieve tailored code for Next.js, Astro, Vite, SvelteKit, etc.

📦 Framework Code Exporters & Auto-Remediation

Generate framework-native TypeScript/JavaScript code for all major stacks:

# Export Next.js App Router integration code
aeo framework nextjs_app --output-dir ./my-next-app

# Or auto-generate concrete code fixes from a live website scan
aeo fix https://mysite.com --framework astro --output-dir ./src/

Supported frameworks:

• Next.js (App Router): app/layout.tsx (with <script type="application/ld+json">), app/robots.ts, app/sitemap.ts, app/llms.txt/route.ts

• Next.js (Pages Router): pages/_document.tsx

• Astro: src/components/AeoHead.astro and astro.config.mjs

• Vite + React / SPA: index.html injection and vite-plugin-aeo.ts

• SvelteKit: src/routes/+layout.svelte and src/routes/robots.txt/+server.ts

• Remix: app/root.tsx meta export

• Nuxt 3: nuxt.config.ts and app.vue useHead()

• Static HTML / Hugo / Jekyll: Clean HTML partials

🚀 Quick Start

1. Installation

# Clone the repository
git clone https://github.com/1nc0gn30/aeo-graph-engine.git
cd aeo-graph-engine

# Editable pip install (zero external runtime dependencies!)
pip install -e .

2. Live Crawl & Audit Any Website

# Crawl live site up to 5 internal pages and compute real AEO scores
aeo scan https://example.com --max-pages 5

Output:

================================================================================
  AEO GRAPH ENGINE — LIVE WEBSITE AUDIT & AI READINESS REPORT
================================================================================
  Target URL:    https://example.com
  Base Origin:   https://example.com
  Overall Score: 78.5 / 100 [GOOD]
  Pages Audited: 5

CATEGORY SCORE BREAKDOWN:
  • Schema Linked Data:     18.0 / 25.0 pts (72%)
  • Llms Txt Machine Index:  0.0 / 25.0 pts (0%)
  • Ai Crawler Governance:  20.0 / 20.0 pts (100%)
  • Semantic Content:       14.0 / 15.0 pts (93%)
  • Technical Seo:          15.0 / 15.0 pts (100%)

ROOT ASSET DISCOVERY:
  [✔ FOUND] robots.txt (HTTP 200)
  [✖ MISSING] llms.txt (HTTP 404)
  [✖ MISSING] llms-full.txt (HTTP 404)
  [✔ FOUND] sitemap.xml (HTTP 200)

AI SEARCH ENGINE BOT ACCESS:
  • OpenAI / ChatGPT (GPTBot):              Allowed [✔ FAQ Schema]
  • Perplexity AI (PerplexityBot):          Allowed [✔ Connected @graph]
  • Anthropic Claude (ClaudeBot):           Allowed
  • Apple Intelligence (Applebot-Extended): Allowed [✔ Software Schema]
  • Google AI Overviews (Google-Extended):  Allowed [✔ Linked Data]

ACTIONABLE OPTIMIZATION PRIORITIES:
  [HIGH] Missing llms.txt standard manifest
  -> Fix: Generate and place a standardized llms.txt at root (llmstxt.org spec) to allow ChatGPT & Perplexity to ingest docs in <50 tokens.

3. Synthesize from Natural Language Prompt

# Auto-configure and write AEO bundle from a natural language prompt
aeo prompt "An AI resume builder called CVForge on cvforge.app" --output-dir dist/

4. Inject Schema into HTML

# Idempotently embed or update Schema.org JSON-LD in your index.html
aeo --inject dist/index.html --niche saas

5. Audit Local Directory

# Audit built directory or single file with full diagnostics
aeo --validate dist/ --format json

💡 Why AEO / GEO Matters in 2026

🛠️ CLI Reference

usage: aeo [-h] [--generate-all] [--output-dir OUTPUT_DIR]
           [--niche {developer_tools,ai_swarm,saas,cybersecurity,spatial_3d,creator,ecommerce,local_business}]
           [--config CONFIG] [--site-name SITE_NAME] [--domain DOMAIN] [--version VERSION]
           [--format {text,json,github}] [--jsonld] [--llms] [--llms-full] [--ai-txt] [--robots]
           [--inject INJECT] [--validate VALIDATE] [--platform] [--test] [--dry-run]
           {init,wizard,scan,bot-audit,compare,report,check,framework,fix,mcp,prompt,serve,ui,extract,platform} ...

Commands & Subcommands

🐍 Python Library API

Import aeo_graph_engine directly into your Python backend, static site generator, or CI script:

from aeo_graph_engine import (
    LiveAEOScanner,
    synthesize_config_from_prompt,
    generate_schema_graph,
    generate_llms_txt,
    generate_robots_txt,
    write_aeo_bundle,
    validate_aeo_bundle
)

# 1. Live crawl and audit any URL
scanner = LiveAEOScanner("https://example.com", max_pages=5)
audit = scanner.compute_audit_scores()
print(f"AEO Score: {audit['overall_aeo_score']}/100")
print(f"Action Items: {len(audit['action_items'])}")

# 2. Synthesize config from natural language
config = synthesize_config_from_prompt("DeFi protocol on Solana called SolarYield")

# 3. Generate Schema.org JSON-LD Graph dictionary
graph = generate_schema_graph(config, niche="saas")

# 4. Write complete bundle and inject into built HTML files
artifacts = write_aeo_bundle(
    output_dir="./dist",
    config=config,
    niche="saas",
    inject_html_files=["./dist/index.html"]
)

# 5. Validate AEO Readiness Score (0-100)
report = validate_aeo_bundle("./dist")
print(f"Readiness Score: {report.score}/100 - Status: {report.to_dict()['status']}")

🤖 AI Crawler Compatibility & Backlink Intelligence

How AI Search Engines Ingest Your Content:

• OpenAI / ChatGPT Search: Combines Bing index with direct queries to llms.txt, robots.txt, and FAQPage JSON-LD schemas.

• Perplexity AI: Discovers URLs via sitemaps, indexes full-text deep research pages (llms-full.txt), and cross-references Wikidata and GitHub entities.

• Anthropic Claude: Extracts knowledge from clean structured markdown representations without visual layout boilerplate, respecting ai.txt attribution policies.

• Google Gemini & AI Overviews: Leverages Knowledge Graph nodes, requiring connected Schema.org @graph linking Organization to WebSite to SoftwareApplication.

High-Authority Knowledge Graph Anchors:

1. llmstxt.org Directory & Awesome-LLMS-Txt: Direct crawler registry for AI documentation indexing.

2. Wikidata & Wikipedia Entity Linking: Primary disambiguation anchor for Google AI Overviews and ChatGPT.

3. GitHub Official Organization & Clean README: Heavily weighted by Claude, Cursor, and Perplexity for technical domain authority.

4. Hugging Face Hub Cards: Indexed continuously by frontier AI web scrapers.

🧩 Built-in Domain Presets

aeo-graph-engine comes with 8 production-tested niche presets:

⚙️ CI / Post-Build Automation

Add an automated AEO step to your build pipeline (Vite, Next.js, Astro, Nuxt, Netlify, GitHub Actions):

package.json Build Hook:

{
  "scripts": {
    "build": "vite build && aeo --generate-all --output-dir dist/ --inject dist/index.html && aeo --validate dist/"
  }
}

GitHub Actions Workflow (.github/workflows/ci.yml):

name: CI & AEO Audit

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - name: Run Tests
        run: |
          pip install -e ".[dev]"
          pytest tests/ -v
          python3 -m aeo_graph_engine.cli --test

📂 Real-World Reference Examples

Explore production-ready starter implementations in the examples/ directory:

• ⚛️ examples/nextjs-app-router/: Complete Next.js 14/15 App Router project with layout.tsx Schema.org injection, dynamic app/robots.ts, app/sitemap.ts, and app/llms.txt/route.ts.

• 🚀 examples/astro-site/: Astro 4/5 integration with reusable <AeoHead /> component, robots.txt, and llms.txt.

• ⚡ examples/vite-react-spa/: Vite React single-page app with custom vite-plugin-aeo that automatically emits machine manifests during vite build.

• ☁️ examples/saas-landing/: Production B2B SaaS configuration preset for multi-agent cloud orchestration.

• 🤖 examples/mcp-clients/: Pre-built MCP integration configs for Claude Desktop, Cursor IDE, Cline, and Zed editor.

See the complete guide in examples/README.md.

🚦 CI/CD Quality Gate & GitHub Actions

Enforce minimum AEO Readiness scores on pull requests and deployments to prevent SEO/AEO regressions:

# Check local bundle or live URL (fails with exit code 1 if score < 85 or llms.txt missing)
aeo check . --min-score 85 --fail-on-missing-llms

# Output GitHub Actions formatted annotations (::error::, ::warning::)
aeo check https://mysite.com --format github

Add to .github/workflows/aeo-audit.yml:

name: AEO Quality Gate

on: [push, pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Install & Audit AEO
        run: |
          pip install -e .
          python3 -m aeo_graph_engine.cli check . --min-score 80 --fail-on-missing-llms --format github

🧪 Testing & Multi-Platform Verification

Run the full automated test suite:

# Run 168 unit tests across engine, crawler, WAF probe, benchmark, reporter, wizard, MCP server, framework exporter, and compat
pytest tests/ -v

# Or run the built-in zero-drift engine test
aeo --test

# Verify your current platform compatibility & environment info
aeo platform

🚀 Automated Release Pipeline

Every push to the main branch or tag creation (v*) triggers the automated release pipeline in .github/workflows/release.yml:

1. Runs full test matrix verification across Linux, macOS, and Windows across Python 3.9–3.13.

2. Builds distribution wheel (.whl) and source archive (.tar.gz).

3. Computes cryptographic SHA-256 checksums (dist/SHA256SUMS.txt).

4. Generates changelog and creates a published GitHub Release with attached assets.

📄 License

Licensed under the MIT License. Extracted and modularized from the Zoth Studio open architecture.