🧭 SAGE — Search, Answer, & Generative Engine Auditor

sage-audit — The Unified 3-Pillar Audit Engine for SEO, Entity AEO, and Generative Engine Optimization (GEO)

Inspect any URL or raw HTML and score it for the era of AI answers — from classical technical SEO to JSON-LD entity graphs to RAG Citation Survival Probability — in one deterministic, crash-proof command.

English · فارسی · Türkçe · Azərbaycan dili · العربية

🇬🇧 English

The Problem: Ten Blue Links Are Dying

Search is undergoing its biggest architectural shift since PageRank. Users no longer click 10 blue links — they get synthesized answers from ChatGPT Search, Perplexity, Claude, Google AI Overviews and Copilot. These engines do not rank pages; they run vector RAG pipelines: they fetch your HTML, strip boilerplate, split it into passages, embed it, retrieve top-k chunks, and let an LLM cite whatever survives.

Classical SEO tools stop at title tags and page speed. They tell you nothing about:

• whether robots.txt is blocking GPTBot, PerplexityBot, ClaudeBot, Google-Extended, Amazonbot or Applebot-Extended;

• whether your JSON-LD entity graph is rich, complete, and anchored to Wikidata/Crunchbase via sameAs;

• whether the first 50–70 words of each section form a direct, fluff-free, factual answer an engine can quote;

• whether your passages are chunked at coherent semantic boundaries (60–120 tokens);

• whether your content survives retrieval — or drowns in semantic entropy and is never cited.

SAGE (sage-audit) closes that gap. One open-source engine, three pillars, four interfaces (CLI · Python library · MCP server · JSON/Markdown exports).

Positioning & Ecosystem Role:
SAGE focuses on static SEO/AEO/GEO diagnostics, analysis, and intelligence extraction (to understand and diagnose). For active multi-page crawling and automated source-code remediation, see SiteProbe (to crawl, analyze, and remediate).

Related MCP server: OctoBoost SEO MCP Server

The 3 Pillars

                    ┌───────────────────────────────────────┐
          L5        │  CITATION SURVIVAL PROXY (CSP) +      │
                    │  llms.txt / rag_ready_chunks.json     │
                    ├───────────────────────────────────────┤
          L4        │  RAG RETRIEVAL SIMULATION             │
                    │  top-k ranking · cosine distance      │
                    ├───────────────────────────────────────┤
          L3        │  VECTOR EMBEDDINGS                    │
                    │  fastembed → sentence-transformers →  │
                    │  deterministic hashed n-gram fallback │
                    ├───────────────────────────────────────┤
          L2        │  SEMANTIC CHUNKING                    │
                    │  60–120 token passage boundaries      │
                    ├───────────────────────────────────────┤
          L1        │  CLEAN DOM EXTRACTION                 │
                    │  boilerplate stripping (Trafilatura)  │
                    └───────────────────────────────────────┘

🏆 Integration with Molavi AI Visibility Index (MAVI) & Benchmark Contribution

sage-audit provides the foundational diagnostic signals (L1–L4) for the Molavi AI Visibility Index (MAVI) and supports the GEO, SEO & Digital Marketing Agency Iran 2026 Benchmark:

• L1 Technical Accessibility: Evaluated via SAGE Technical Auditor (robots.txt AI crawler policies for GPTBot, PerplexityBot, ClaudeBot; clean DOM extraction; HTTP security headers).

• L2 Semantic Extractability: Evaluated via SAGE semantic passage chunking (60–120 token boundaries) and clean text-to-code ratio.

• L3 Entity Clarity: Evaluated via SAGE AEO JSON-LD entity graph validation (Organization, FAQPage, sameAs Wikidata/Crunchbase disambiguation).

• L4 Citation Readiness: Evaluated via SAGE Citation Survival Proxy (CSP) using in-memory RAG retrieval simulation and semantic entropy scoring.

• L5 Empirical AI Visibility: Measured by GEO-Scope across real multi-model provider executions.

• Ecosystem Architecture: See Benchmark Ecosystem Map for how SAGE diagnostics feed directly into MAVI composite scoring and SiteProbe automated remediation.

🏛️ Ecosystem

SAGE Audit operates as the static diagnostic component of the Molavi AI Visibility Stack:

• Discovery: AnswerPath GEO

• Measurement: GEO-Scope

• Diagnostics: SAGE Audit (understand & diagnose)

• Action: SiteProbe (crawl & remediate)

• Protocol: MCP GEO Server

📖 Runnable Python Example

Run the bundled 3-pillar audit script:

python examples/audit_example.py

A complete JSON audit output sample is available at examples/example_audit.json.

Installation

Install from Source / Repository (Pre-release)

# Standard installation from GitHub
pip install git+https://github.com/tmolavi/sage-audit.git

# Recommended: with local neural embeddings for the GEO pillar
pip install "sage-audit[embeddings] @ git+https://github.com/tmolavi/sage-audit.git"

# With MCP server for Claude Desktop / Cursor / AI agents
pip install "sage-audit[mcp] @ git+https://github.com/tmolavi/sage-audit.git"

# Everything at once
pip install "sage-audit[all] @ git+https://github.com/tmolavi/sage-audit.git"

PyPI: Install the latest stable release directly with pip install sage-audit.

Zero-crash guarantee: if no neural embedding package is installed (or a model download fails), the GEO pillar silently falls back to a deterministic hashed n-gram TF vectorizer. Results stay stable and reproducible — the tool never crashes.

Quickstart (CLI)

sage audit https://your-site.com
sage audit https://your-site.com --format markdown -o report.md --save-artifacts ./out
sage audit --raw-html page.html --format json | jq .overall_score
sage generate-llms https://your-site.com -o llms.txt --also-chunks
sage audit https://your-site.com --fail-under 70    # CI quality gate → exit 1

Sample (rich) terminal output:

╭────────────────────────────── SAGE AUDIT REPORT ──────────────────────────────╮
│  https://acme-analytics.example.com/                                          │
│  2026-09-01 09:45 UTC · 9 ms · sage-audit v1.0.0                              │
│  ████████████████████░░░░  84.8/100  (Grade B)                                │
╰────────────────────────────────────────────────────────────────────────────────╯
Pillar 1 — Technical SEO            —  96.9/100 (Grade A+)
│ Canonical URL           ✔ pass  │ canonical → https://acme-analytics.example.com
│ Meta robots directives  ✔ pass  │ content="index,follow"
│ robots.txt AI-crawlers  ⚠ warn  │ GPTBot: blocked; ClaudeBot: allowed; …
Pillar 2 — Answer Engine Optimization (AEO) —  90.5/100 (Grade A)
│ Entity authority        ✔ pass  │ 5 sameAs URL(s); anchors: GitHub, LinkedIn,
│                                 │ Wikidata, Wikipedia, X/Twitter
│ Direct-answer density   ✔ pass  │ 5 sections · avg window score 0.74 ·
│                                 │ 5/5 open with an explicit answer
Pillar 3 — Generative Engine Optimization (GEO) —  68.6/100 (Grade C)
│ Chunk size compliance   ✔ pass  │ 100% of chunks inside 60–120 tokens
│ Citation Survival       ✖ fail  │ CSP 43.1% across 8 simulated RAG queries
│ Semantic retrieval      ⚠ warn  │ entropy 0.67 — passages look alike
╭────────────────────── Priority recommendations ───────────────────────╮
│ ✖ Citation Survival Probability (GEO)                                 │
│     Retrieval attention is diffuse — consolidate per-topic sections   │
│     and front-load definitional sentences.                            │
╰─────────────────────────────────────────────────────────────────────────╯
Artifacts generated: llms.txt, rag_ready_chunks.json

Python API (Library)

from sage_audit import SageAuditor

auditor = SageAuditor(embedding_backend="auto", top_k=3)

# Live URL — fetches the page AND its robots.txt
report = auditor.audit("https://molavi.pro")

# Raw HTML — fully offline
report = auditor.audit_html(html_source, url="https://example.local/")

print(report.overall_score, report.grade)          # e.g. 84.8 'B'
print(report.seo.score, report.aeo.score, report.geo.score)
print(report.artifacts["llms.txt"])                # ready-to-serve manifest

for finding in report.geo.findings:
    if finding.recommendation:
        print(finding.title, "→", finding.recommendation)

MCP Server (Claude Desktop · Cursor · Agents)

SAGE ships a FastMCP server exposing four autonomous tools: sage_audit_url, sage_audit_html, sage_generate_llms_txt, sage_version.

sage mcp                       # stdio transport (default)
sage mcp --transport sse --host 127.0.0.1 --port 8642

Register it in claude_desktop_config.json:

{
  "mcpServers": {
    "sage-audit": {
      "command": "sage",
      "args": ["mcp"]
    }
  }
}

Or without installing, via uvx:

{
  "mcpServers": {
    "sage-audit": {
      "command": "uvx",
      "args": ["--from", "sage-audit[mcp]", "sage", "mcp"]
    }
  }
}

🔬 Evidence & Diagnostic Benchmarks

• Methodology and reproducibility: docs/methodology.md

• Sample Diagnostic Audit Output: examples/example_audit.json

• Standalone Offline Diagnostic Demo: examples/public_demo/

• Mathematical Specification & Taxonomy: docs/methodology.md

• Ecosystem Data Flow & Contracts: docs/BENCHMARK_ECOSYSTEM.md

• Cross-Repository Evidence Map: Ecosystem Evidence Flow

📊 Benchmark & Reproducibility Protocol

To prevent non-reproducible AI benchmark claims and account for evaluation stochasticity, SAGE enforces a formal statistical reproducibility pipeline (see docs/methodology.md):

• Minimum $N \ge 20$ Repeated Runs: Any public comparative claim requires at least 20 repeated runs (--public-claim), while fast single runs remain available for local development and CI testing.

• Statistical Rigor: Computes 95% Wilson Score Confidence Intervals for success rates and 95% non-parametric bootstrap confidence intervals ($B=2000$) for latency. If confidence intervals overlap, claims of superiority are disallowed.

• 18-Field Telemetry & Provenance: Every observation records 18 standard fields (benchmark_id, task_id, run_id, pass, raw_score, model, provider, model_version_if_available, framework_or_agent_version, temperature, timestamp, latency_ms, input_tokens, output_tokens, estimated_cost_usd, tool_calls, config_hash, git_sha, raw_evidence_path).

• Complete Run Preservation: Artifacts are saved under results/<benchmark>/<run-date>/ (manifest.json, observations.jsonl, summary.csv), ensuring summary.csv is 100% mathematically recoverable from observations.jsonl.

• 1-Command Reproduction: Any researcher can verify or re-run a benchmark from its pinned manifest.

# Run benchmark with public claim validation (20 runs + Wilson/Bootstrap CIs)
sage benchmark run --suite resource --runs 20 --public-claim

# Reproduce any past benchmark run from its pinned manifest
sage benchmark reproduce results/resource_eval_public_test/2026-10-02/manifest.json

Scoring Model & Epistemic Evidence Taxonomy

• Every check is a weighted Finding tagged with structured epistemic Evidence Taxonomy (E0–E5) metadata:

  • E0: Deterministic technical facts (HTTP status, headers, HTML tag presence, syntax).

  • E1: Standards & specifications (W3C HTML5, Schema.org, IETF RFCs).

  • E2: Documented search engine & platform guidance (Google Search Central, bot policies).

  • E3: Empirical evidence from validated benchmark datasets.

  • E4: Industry heuristics & configurable thresholds (word counts, chunk sizes, answer density).

  • E5: Experimental proxies & mathematical models (CSP, softmax semantic entropy).

• Prohibition of Unproven Claims: SAGE strictly labels heuristics and proxies without asserting unproven direct ranking-factor claims.

• Pillar score = weighted mean × 100. Overall = 0.30·SEO + 0.35·AEO + 0.35·GEO (configurable).

• Grades: A+ ≥ 93 · A ≥ 85 · B ≥ 75 · C ≥ 65 · D ≥ 50 · F < 50.

• Citation Survival Proxy (CSP): A heuristic proxy score (0–100) evaluating candidate passage retrieval prominence and low semantic entropy under simulated dense vector retrieval: $$\text{CSP} = \text{mean}(0.6 \cdot \text{prominence} + 0.4 \cdot (1 - \text{entropy})) \times 100$$ (Note: CSP is an uncalibrated heuristic retrieval proxy, not a literal probability of external AI citation).

• Empirical Validation Framework: Compare SAGE/CSP scores against real observed AI search citation logs via Spearman correlation ($\rho$), AUROC, Precision@k, Brier score, and probability calibration curves (sage validate).

• Detailed specification in docs/methodology.md.

Project Structure

sage-audit/
├── LICENSE                     # MIT — Copyright (c) 2026 Taqi Molavi
├── pyproject.toml              # build system, metadata, deps, `sage` entrypoint
├── requirements.txt
├── .gitignore
├── docs/
│   ├── methodology.md          # Full Evidence Taxonomy & CSP specification
│   ├── reproducibility-protocol.md # Statistical benchmark protocol (Wilson/Bootstrap CI)
│   └── BENCHMARK_ECOSYSTEM.md  # Ecosystem data flow & contracts
├── results/                    # Reproducible benchmark manifests & observations
├── src/
│   └── sage_audit/
│       ├── __init__.py         # exports SageAuditor, evaluation, validation, __version__
│       ├── _version.py         # single source of truth
│       ├── config.py           # SageConfig & configurable heuristic thresholds
│       ├── models.py           # pydantic schemas (EvidenceMetadata, CspDetails, Reports)
│       ├── evaluation.py       # Benchmark runner, 18-field telemetry, Wilson/Bootstrap CI
│       ├── memory_eval.py      # SAGE Memory benchmark suite
│       ├── validation.py       # Spearman, AUROC, Precision@k, Brier score, calibration
│       ├── cli.py              # sage audit | generate-llms | validate | benchmark | mcp
│       ├── core.py             # SAGE orchestrator (0.30/0.35/0.35 configurable fusion)
│       ├── auditors/
│       │   ├── __init__.py
│       │   ├── seo_auditor.py  # Pillar 1 — Technical SEO (E0–E4)
│       │   ├── aeo_auditor.py  # Pillar 2 — Entity AEO (E1–E4)
│       │   └── geo_auditor.py  # Pillar 3 — GEO / RAG simulation / CSP (E0, E4, E5)
│       ├── server/
│       │   ├── __init__.py
│       │   └── mcp_server.py   # FastMCP tools for AI agents
│       └── utils/
│           ├── __init__.py
│           ├── extractor.py    # clean text / DOM parser (Trafilatura + BS4)
│           ├── formatter.py    # terminal / JSON / Markdown renderers with taxonomy
│           └── text.py         # tokenizer, sentence splitter, vector math
├── tests/
│   ├── __init__.py
│   ├── test_auditors.py        # SEO, AEO, GEO, core, CLI unit tests
│   ├── test_taxonomy.py        # Evidence taxonomy & ranking claims tests
│   ├── test_csp_hardening.py   # Hardened CSP proxy semantics tests
│   ├── test_validation.py      # Statistical validation & calibration tests
│   ├── test_reproducibility_protocol.py # 20-run, Wilson/Bootstrap CI, manifest pinning tests
│   └── test_config.py          # Heuristic threshold override tests
└── README.md

Development

git clone https://github.com/tmolavi/sage-audit.git
cd sage-audit
pip install -e ".[dev]"
pytest -q

📦 Distribution & Publishing

Two "publish" targets — don't confuse them

For contributors and unreleased changes, SAGE can also be installed directly from the repo:

pip install git+https://github.com/tmolavi/sage-audit.git
# with every optional extra (embeddings + mcp):
pip install "sage-audit[all] @ git+https://github.com/tmolavi/sage-audit.git"

Path A — PyPI release maintenance

GitHub and PyPI are separate services: pushing code to GitHub never publishes a pip package. To put sage-audit on PyPI the maintainer must:

1. Create a PyPI account at pypi.org/account/register, verify the email address, and enable two-factor authentication (PyPI refuses uploads from accounts without 2FA).

2. Mint an API token at pypi.org/manage/account/token → Add API token.

   • For routine releases, use a short-lived project-scoped token or the repository's Trusted Publishing workflow.

   • The token starts with pypi- — treat it like a password and never commit it to git.

3. Build & upload from the repo root:

   pip install --upgrade build twine
   python -m build        # → dist/sage_audit-<version>-py3-none-any.whl + .tar.gz
   twine upload dist/*    # username: __token__    password: pypi-...

4. Verify at pypi.org/project/sage-audit — from that moment pip install sage-audit works worldwide.

5. Hygiene — revoke the token immediately after the upload (or create a new project-scoped one and keep the expiry short).

Path B — Trusted Publishing (recommended, zero secrets)

This repository already ships .github/workflows/publish.yml, which uses the official pypa/gh-action-pypi-publish action over GitHub OIDC — no token is ever stored anywhere.

The PyPI-side project publisher is already configured with:

• Owner: tmolavi · Repository: sage-audit

• Workflow filename: publish.yml

• GitHub environment: pypi

To publish a new version, update the package version, push the changes, and cut a release on GitHub (Releases → Draft a new release → tag the matching version): the workflow builds the wheel + sdist and publishes them automatically — every future release is a one-click PyPI publish.

GitHub Topics

seo · aeo · geo · generative-engine-optimization · ai-search · rag · mcp-server · perplexity · chatgpt-search · llms-txt · schema-markup · entity-seo

🇮🇷 فارسی

SAGE — سِیج: اولین موتور ممیزی متن‌باز سه‌گانهٔ سئو، AEO و GEO در دنیا

sage-audit (مخفف Search, Answer, & Generative Engine Auditor) اولین ابزار متن‌باز جهان است که هر سه ستون بهینه‌سازی برای موتورهای جست‌وجو، موتورهای پاسخ‌گو و موتورهای مولد را در یک دستور واحد ممیزی می‌کند. توسعه‌دهنده و معمار این پروژه تقی مولوی است.

چرا SAGE؟

دوران ده لینک آبی تمام شده است. کاربران امروز پاسخ خود را از ChatGPT، Perplexity، Claude و Google AI Overviews می‌گیرند؛ سیستم‌هایی که صفحهٔ شما را «رتبه‌بندی» نمی‌کنند، بلکه آن را به قطعات برداری (Chunk) تقسیم کرده و از طریق خط‌لولهٔ RAG بازیابی می‌کنند. ابزارهای کلاسیک سئو هیچ‌چیز دربارهٔ این‌ها نمی‌گویند:

• سیاست robots.txt در برابر خزنده‌های هوش مصنوعی (GPTBot، PerplexityBot، ClaudeBot، Google-Extended، Amazonbot، Applebot-Extended)؛

• عمق گراف موجودیت JSON-LD و لنگرهای اعتبار sameAs (ویکی‌دیتا، ویکی‌پدیا، Crunchbase)؛

• چگالی «پاسخ مستقیم» در ۵۰ تا ۷۰ کلمهٔ نخست هر بخش؛

• و احتمال بقای ارجاع (Citation Survival Probability) در خط‌لولهٔ بازیابی برداری.

معماری هرم ۵ لایهٔ GEO

          لایهٔ ۵  │  احتمال بقای ارجاع + تولید llms.txt و rag_ready_chunks.json
          لایهٔ ۴  │  شبیه‌سازی بازیابی RAG (رتبه‌بندی top-k با شباهت کسینوسی)
          لایهٔ ۳  │  تعبیهٔ برداری: fastembed ← sentence-transformers ← بردارساز قطعی n-gram
          لایهٔ ۲  │  قطعه‌بندی معنایی متن (مرزهای ۶۰ تا ۱۲۰ توکن)
          لایهٔ ۱  │  استخراج DOM تمیز و حذف صفحه‌آرایی (Trafilatura)

یکی از خروجی‌های خودکار، فایل llms.txt است: سندی استاندارد که به LLMها اعلام می‌کند صفحهٔ شما چه ساختاری دارد — SAGE آن را همراه با rag_ready_chunks.json به‌صورت خودکار تولید می‌کند.

نصب و اجرای سریع

pip install sage-audit
pip install "sage-audit[embeddings]"   # توصیه‌شده: تعبیهٔ عصبی محلی
pip install "sage-audit[mcp]"          # سرور پروتکل MCP

sage audit https://molavi.pro
sage audit https://molavi.pro --format markdown -o report.md --save-artifacts ./out
sage generate-llms https://molavi.pro -o llms.txt --also-chunks
sage audit https://molavi.pro --fail-under 70   # دروازهٔ کیفی در CI
sage mcp                                        # اتصال به Claude Desktop / Cursor

📊 پروتکل بازتولیدپذیری و بنچ‌مارک‌های آماری SAGE

برای جلوگیری از ادعاهای غیرقابل اثبات و نوسانات تصادفی مدل‌های هوش مصنوعی، موتور SAGE به یک روش استاندارد بازتولیدپذیری و اعتبارسنجی آماری مجهز است (مستندات روش‌شناسی):

• حداقل ۲۰ تکرار برای ادعاهای عمومی (--public-claim): هرگونه ادعای مقایسه‌ای عمومی مستلزم حداقل ۲۰ بار اجرای تکراری مستقل است؛ در حالی که برای تست‌های سریع توسعه اجرای تک‌باره همچنان مجاز است.

• دقت آماری (فاصله‌های اطمینان ۹۵٪ ویلسون و بوت‌استرپ): نرخ موفقیت با فاصلهٔ اطمینان ۹۵٪ Wilson Score و توزیع تأخیر با فاصله اطمینان ۹۵٪ بوت‌استرپ ناپارامتریک محاسبه می‌شود. در صورت هم‌پوشانی بازه‌های اطمینان، برتری قاطعی اعلام نمی‌شود.

• ثبت شواهد خام ۱۸ فیلدی و مانیفست تبارشناسی: تمام مشاهدات در ساختار results/<benchmark>/<date>/ با فرمت‌های manifest.json، observations.jsonl و summary.csv ذخیره شده و امکان استخراج و بررسی ۱:۱ دارند.

• دستور سادهٔ بازتولید (Reproduction):

# اجرای بنچ‌مارک با اعتبارسنجی آماری ادعای عمومی (۲۰ تکرار)
sage benchmark run --suite resource --runs 20 --public-claim

# بازتولید و اعتبارسنجی یک اجرای گذشته از طریق فایل مانیفست
sage benchmark reproduce results/resource_eval_public_test/2026-10-02/manifest.json

برای تغییرات توسعه‌ای و نسخه‌های منتشرنشده، می‌توانید مستقیماً از ریپو نصب کنید:

pip install git+https://github.com/tmolavi/sage-audit.git

انتشار روی PyPI (اختیاری — راهنمای گام‌به‌گام نگهدارنده)

دو مقصد انتشار را با هم اشتباه نکنید: گیت‌هاب (سورس‌کد) و PyPI (پکیج pip روی pypi.org). پوش کردن کد به گیت‌هاب به‌تنهایی پکیج pip منتشر نمی‌کند؛ برای انتشار، یک GitHub Release منتشرشده بسازید.

تا قبل از اولین انتشار رسمی، هر کسی می‌تواند مستقیم از ریپو نصب کند:

pip install git+https://github.com/tmolavi/sage-audit.git

برای انتشار رسمی روی PyPI، روش پیشنهادی Trusted Publishing است:

1. ساخت اکانت PyPI — در pypi.org/account/register ثبت‌نام کنید، ایمیل را تأیید و احراز هویت دو مرحله‌ای (2FA) را فعال کنید (بدون 2FA پلتفرم اجازهٔ آپلود نمی‌دهد).

2. ساخت API Token — در pypi.org/manage/account/token روی Add API token بزنید. چون پروژه هنوز روی PyPI وجود ندارد، scope اولین توکن باید «Entire account» باشد (توکن پروژه‌محور فقط بعد از اولین آپلود قابل ساخت است). توکن با pypi- شروع می‌شود و مثل رمز عبور محرمانه است.

3. Build و آپلود — از ریشهٔ پروژه:

   pip install --upgrade build twine
   python -m build        # خروجی: dist/sage_audit-1.0.0-py3-none-any.whl و .tar.gz
   twine upload dist/*    # username: __token__ — password: همان توکن pypi-...

4. کنترل نهایی — صفحهٔ pypi.org/project/sage-audit را باز کنید؛ از این لحظه pip install sage-audit در سراسر جهان کار می‌کند.

5. امنیت — بلافاصله بعد از آپلود، توکن را Revoke کنید (یا یک توکن جدید پروژه‌محور با تاریخ انقضای کوتاه بسازید). توکن را هرگز داخل git کامیت نکنید.

فایل آمادهٔ .github/workflows/publish.yml از GitHub OIDC استفاده می‌کند. Publisher پروژه در PyPI به Owner: tmolavi، Repo: sage-audit، Workflow: publish.yml و Environment: pypi متصل است. از این به بعد هر GitHub Release منتشرشده، به‌طور خودکار روی PyPI منتشر می‌شود، بدون اینکه هیچ توکنی در GitHub ذخیره شود.

تضمین پایداری

اگر بستهٔ تعبیهٔ عصبی نصب نباشد یا دانلود مدل شکست بخورد، ستون GEO به‌صورت خودکار به بردارساز قطعی و بازتولیدپذیر n-gram کاهش می‌یابد؛ ابزار هرگز متوقف نمی‌شود و نتایج همواره قطعی و قابل اتکا می‌مانند.

توسعه‌دهنده

🇹🇷 Türkçe

SAGE Projesi: SEO, AEO ve GEO için Dünyanın İlk Birleşik Açık Kaynak Denetim Motoru

sage-audit (Search, Answer, & Generative Engine Auditor), bir web sayfasını üç sütunda birden denetleyen dünyanın ilk birleşik açık kaynak motorudur: klasik teknik SEO, varlık tabanlı AEO (Cevap Motoru Optimizasyonu) ve GEO (Üretken Motor Optimizasyonu). Projenin geliştiricisi ve mimarı Taqi Molavi'dir.

Neden SAGE?

Mavi bağlantılar çağı kapanıyor. Kullanıcılar artık yanıtları ChatGPT, Perplexity, Claude ve Google AI Overviews gibi sistemlerden alıyor. Bu sistemler sayfanızı sıralamaz; HTML'inizi alır, metni 60–120 tokenlık parçalara böler, vektör uzayına gömer ve yalnızca RAG hattından sağ çıkan pasajları alıntılar. SAGE şunları ölçer:

• robots.txt dosyasının yapay zekâ tarayıcılarına (GPTBot, PerplexityBot, ClaudeBot, Google-Extended, Amazonbot, Applebot-Extended) karşı politikası;

• JSON-LD varlık grafiğinin derinliği ve Wikidata / Wikipedia / Crunchbase gibi sameAs otorite sinyalleri;

• Her bölümün ilk 50–70 kelimesindeki doğrudan yanıt yoğunluğu;

• Kosinüs uzaklığı + anlamsal entropi ile Alıntı Hayatta Kalma Olasılığı (CSP);

• 5 katmanlı GEO piramidi sonunda otomatik olarak llms.txt ve rag_ready_chunks.json üretimi.

Kurulum ve Hızlı Başlangıç

pip install sage-audit
pip install "sage-audit[embeddings]"   # önerilen: yerel sinirsel gömme
pip install "sage-audit[mcp]"          # MCP sunucusu

sage audit https://ornek-siteniz.com
sage audit https://ornek-siteniz.com --format markdown -o rapor.md --save-artifacts ./cikti
sage generate-llms https://ornek-siteniz.com -o llms.txt --also-chunks
sage audit https://ornek-siteniz.com --fail-under 70   # CI kalite kapısı
sage mcp                                               # Claude Desktop / Cursor bağlantısı

📊 SAGE Karşılaştırmalı Değerlendirme ve Yeniden Üretilebilirlik Protokolü

Tek çalıştırmalı testlerdeki gürültüyü ve dayanaksız karşılaştırmalı iddiaları önlemek amacıyla SAGE, istatistiksel olarak doğrulanabilir bir değerlendirme hattı sunar (ayrıntılı metodoloji):

• Kamuya Açık İddialar İçin Minimum 20 Tekrar (--public-claim): Karşılaştırmalı tüm kamu iddiaları en az 20 bağımsız çalıştırma gerektirir; yerel geliştirmede ise hızlı tek çalıştırmalar desteklenir.

• İstatistiksel Güven Aralıkları (%95 Wilson ve Bootstrap CI): Başarı oranı %95 Wilson Skor Güven Aralığı ile, gecikmeler ise parametrik olmayan bootstrap (%95 CI) ile hesaplanır. Güven aralıkları örtüşüyorsa kesin üstünlük ilan edilmez.

• 18 Alanlı Ham Kanıt Şeması ve Tam Köken Takibi: Her gözlem results/<benchmark>/<tarih>/ dizininde manifest.json, observations.jsonl ve summary.csv olarak kaydedilir.

• Tek Komutla Yeniden Üretim:

# Kamuya açık iddia protokolü ile 20 tekrarlı çalıştırma
sage benchmark run --suite resource --runs 20 --public-claim

# Manifest dosyası üzerinden geçmiş çalıştırmayı yeniden üretme
sage benchmark reproduce results/resource_eval_public_test/2026-10-02/manifest.json

Geliştirme değişiklikleri ve yayımlanmamış sürümler için doğrudan depodan kurabilirsiniz:

pip install git+https://github.com/tmolavi/sage-audit.git

PyPI'de Yayınlama (İsteğe Bağlı — Adım Adım Kılavuz)

İki yayın hedefini karıştırmayın: GitHub (kaynak kodu) ve PyPI (pypi.org'daki pip paketi). GitHub'a push etmek tek başına bir pip paketi yayınlamaz; yayımlanmış bir GitHub Release oluşturmanız gerekir.

İlk resmî PyPI sürümünden önce herkes doğrudan depodan kurabilir:

pip install git+https://github.com/tmolavi/sage-audit.git

Resmî PyPI yayını için:

1. PyPI hesabı — pypi.org/account/register adresinden kaydolun, e-postayı doğrulayın ve 2FA'yı etkinleştirin (2FA olmadan yükleme yapılamaz).

2. API tokenı — pypi.org/manage/account/token sayfasında Add API token. Proje henüz PyPI'de olmadığı için ilk tokenın kapsamı «Entire account» olmalıdır (proje bazlı token ancak ilk yüklemeden sonra oluşturulabilir). Token pypi- ile başlar ve parola gibi gizli tutulmalıdır.

3. Derleme & yükleme — depo kökünden:

   pip install --upgrade build twine
   python -m build        # çıktı: dist/sage_audit-1.0.0-py3-none-any.whl + .tar.gz
   twine upload dist/*    # kullanıcı adı: __token__ — parola: pypi-... tokenı

4. Doğrulama — pypi.org/project/sage-audit sayfasını açıp pip install sage-audit komutunu test edin; artık tüm dünyada çalışır.

5. Güvenlik — yüklemeden hemen sonra tokenı revoke edin (ya da yalnızca bu projeye özel, kısa süreli bir token oluşturun). Tokenı asla git'e commit etmeyin.

Trusted Publishing (önerilen, tokensiz): depoda hazır bulunan .github/workflows/publish.yml dosyası GitHub OIDC kullanır. PyPI Publisher ayarı tmolavi/sage-audit (workflow: publish.yml, environment: pypi) olarak yapılandırılmıştır; bundan sonraki her yayımlanmış GitHub Release otomatik olarak PyPI'de yayınlanır.

Geliştirici

SAGE; sinirsel gömme modeli bulunamadığında otomatik olarak deterministik n-gram vektörleştiriciye geçer — araç asla çökmez, sonuçlar her zaman tekrar üretilebilirdir.

💬 Community & External Collaboration

We welcome researchers and engineers contributing to multi-pillar diagnostic algorithms and evidence taxonomies:

• Discussions: GitHub Discussions across General, Research, Ideas, Help, and Show & Tell.

• First Contribution Guide: docs/FIRST_CONTRIBUTION.md

• Methodology & CSP Formalization: docs/methodology.md

• Report Issues: GitHub Issues

• Contribution Standards: CONTRIBUTING.md and SECURITY.md

Author & Architect

Taqi Molavi (Taghi Molavi)

🌐 molavi.pro · 🔬 Molavi R&D Think Tank — molavi.pro/research · 🐙 github.com/tmolavi

If SAGE helps your research or your rankings, please star ⭐ the repository — it keeps the engine open-source and evolving.

License: MIT — Copyright (c) 2026 Taqi Molavi