Shayari: Devanagari-first shayari search MCP

A small, reproducible Python package for searching and reading Hindi/Devanagari shayari. The user-facing language is Hindi. Urdu is not required for search or reading, and this project does not transliterate or generate missing Hindi text.

Status

The first milestone uses the approved Kaggle source:

• Urdu Ghazals Rekhta

• Kaggle version 1, downloaded without credentials

• 1,314 source stems, 30 poets

• 1,306 Hindi files, of which one is empty

• 1,305 source-valid Hindi-text records are active by explicit owner authorization. Nine source stems remain quarantined because they have no usable Hindi text.

The source metadata declares CC0, but the underlying Rekhta-derived rights are not independently verified. Read corpus/SOURCE.md before redistribution.

Related MCP server: video-evidence-mcp

Design decisions

• text_devanagari is the canonical field and is required for active records.

• No Urdu-to-Devanagari transliteration, translation, repair, or generation occurs.

• Invalid or incomplete records are written to quarantine, never silently indexed.

• title_devanagari and poet_devanagari remain optional because the inspected archive supplies only Roman source-path metadata.

• title_source and poet_source preserve that metadata explicitly; it is searchable but is not mislabeled as Hindi.

• Downloaded archives are ignored by Git. The generated active snapshot is versioned so a fresh checkout can search immediately; it contains no credentials.

Requirements

• Python 3.9+

• pytest for tests

• Optional: the mcp Python package for the standard MCP server adapter

The core importer, validator, search service, and CLI use only the Python standard library.

Reproduce the corpus

From the project root, place the downloaded archive at .cache/urdu-ghazals-rekhta.zip, then run:

PYTHONPATH=src python3 scripts/import_kaggle.py .cache/urdu-ghazals-rekhta.zip

The command writes these files under corpus/generated/:

• records.jsonl: active, owner-authorized source-valid records. Every record retains explicit provenance and authorization-status fields.

• research.jsonl: empty under the default owner-authorized policy; available when using --activation-policy research_only for comparison.

• quarantine.jsonl: rejected or incomplete source stems with reasons

• audit.json: counts, source version, checksum, script distribution, and activation policy

To audit an existing generated directory:

PYTHONPATH=src python3 scripts/audit_corpus.py corpus/generated

CLI

Search the active, owner-authorized Hindi Devanagari corpus:

PYTHONPATH=src python3 -m shayari search 'दोस्ती' --limit 5
PYTHONPATH=src python3 -m shayari search 'मोहब्बत' --limit 5

The default importer policy is owner_authorized, which activates every source-valid Devanagari record while preserving the source's provenance and limitations. The activation is an owner decision, not a claim of independent legal or linguistic verification. To reproduce the earlier research-only layout explicitly:

PYTHONPATH=src python3 scripts/import_kaggle.py .cache/urdu-ghazals-rekhta.zip --activation-policy research_only

The current source has 1,305 active records after import. The remaining nine source stems stay quarantined because they have missing or empty Hindi text.

PYTHONPATH=src python3 -m shayari status

Set SHAYARI_CORPUS if the generated corpus lives outside corpus/generated:

SHAYARI_CORPUS=/path/to/generated shayari status

The explicitly named research-only MCP tool is available for inspection.

Search title and poet fields too, and inspect a result by ID:

PYTHONPATH=src python3 -m shayari show <record-id>
PYTHONPATH=src python3 -m shayari audit

All normal search results include the full text_devanagari field.

MCP server

The package exposes a stdio server through either the shayari-mcp command or the module entry point. With the optional mcp dependency installed:

python3 -m pip install 'mcp>=1.0,<2.0'
PYTHONPATH=src python3 -m shayari.mcp_server
# or, after installing this project:
shayari-mcp

The server tools are:

• search_shayari

• search_research_shayari (explicitly non-active research inspection)

• get_shayari

• audit_corpus

The handlers call the same lexical service as the CLI and return Hindi-facing fields by default. An in-process JSON-RPC-compatible fallback is included for local clients that do not have the optional SDK installed.

Example Hermes MCP configuration:

mcp_servers:
  shayari:
    command: python3
    args: ["-m", "shayari.mcp_server"]
    env:
      PYTHONPATH: "/absolute/path/to/Shayari/src"

Development

PYTHONPATH=src python3 -m pytest -q
ruff check src scripts tests
ruff format --check src scripts tests
mypy --python-version 3.9 src scripts
python3 -m build --wheel --sdist

The downloaded .cache/ archive is ignored. The versioned corpus/generated/ snapshot can be regenerated from it. Do not add credentials, tokens, or environment files to the repository.