Reads system-prompt.md, contract.md, skill.md and every *-context.md from
agents/{agent_slug}/. The context files carry the project-specific detail —
methodology parameters, pipeline inventories, past decisions — that turns a
generic specialist into one that knows this researcher's actual work.
If the agent is not found, lists all available agents.

Args:
    agent_slug: Folder name of the agent (e.g. "archivist", "librarian").

Records that an agent did a piece of work so it appears in the dashboard's
Agents view and in get_agent_runs. Call it after writing an output file, per
the output contract. When a session_id is supplied it also writes a "result"
event to session_events, closing the loop for /metis pipeline calls.

Args:
    agent_slug: Slug of the agent that performed the work (e.g. "librarian").
    task_summary: Brief description of what the agent did.
    input_path: Path to the input file(s), if any (default empty string).
    output_path: Path to the output file(s) produced, if any (default empty).
    complexity: The run status stored in the `status` column — typically
        "completed", "partial", or "failed" (default "standard").
    input_tokens: Input tokens consumed, for cost tracking (default 0).
    output_tokens: Output tokens produced, for cost tracking (default 0).
    model: Model identifier used, e.g. "claude-sonnet-4-6" (default empty).
    session_id: Pipeline session ID from session_bootstrap(); when set, also
        records a result event in session_events (default empty string).

Returns:
    A confirmation message naming the agent and task that were logged.

Returns the log of past agent work — what ran, when, its status, and token
usage — for the dashboard or for reviewing recent activity. These rows are
written by log_agent_run. Results come back newest first.

Args:
    limit: Maximum number of runs to return, newest first (default 10).
    since: ISO date or datetime string; only runs at or after this time are
        returned. Empty string (default) returns runs from all dates.
    agent_slug: Filter to a single agent by slug, e.g. "librarian" or
        "metis". Empty string (default) returns all agents.

Returns:
    A text block listing the matching runs (run_id, agent, task summary,
    status, timestamp, token counts, model).

Searches your curated/seeded literature catalogue by structured facets
(disease, method, geography, keyword). For your Zotero/manual reference
metadata use search_library; for full PDF body text use search_fulltext;
for semantic RAG over indexed PDFs use search_pdf_knowledge.

Searches the library_seeded SQLite table. Use this to find papers
by disease focus, methodology, geography, or any keyword.

Args:
    query: Search term, matched as a case-insensitive substring.
    field: Column to search -- one of "all", "disease", "method",
        "geography", or "article".
    limit: Maximum number of results to return (default 20).

Returns:
    A single TextContent holding a markdown table of matching rows from the
    library_seeded table, or an error/"no results" message if the database,
    table, or column is missing or nothing matches.

Case-insensitive substring search with surrounding context lines.

Args:
    query: Search term.
    scope: Where to search -- "all", "domains", "projects", "library".
    limit: Maximum results (default 15).
    max_chars_per_result: Truncate each result's context to this many characters (default 500).
                          Pass 0 for no truncation.

Called when a researcher confirms they want a Claude conversation or project
tracked permanently in Metis. Creates the project record in the DB so it
appears in the Work tab and is available for task linking and memory search.

Args:
    title: Human-readable project name, e.g. "Statistics Course".
    description: What this project is about (one sentence).
    domain: Research domain, e.g. "education", "epidemiology". Optional.
    source: Origin — "claude_project" (default), "claude_cowork", or "manual".

Reads the project REGISTRY (the `projects` table — the source of truth that
the dashboard Work tab shows), not just the folders on disk, and adds task
completion counts. Empty project_id lists ALL active projects; a specific
project_id (exact or partial) shows that one, enriched from its folder card
if a `projects/active/<name>/` folder exists.

Args:
    project_id: Project id (or part of one). Empty string = all active projects.

Sets status='archived' in projects table. Project disappears from
active view but remains available for brainstorm context and search.

Args:
    project_id: The project_id to archive.

Args:
    project_id: The project_id to restore.

Deletes project record and associated tasks from DB.
If delete_files=True, also deletes the external_path folder from disk
(only if path is within RC root — safety check enforced).

Args:
    project_id: The project_id to remove.
    delete_files: If True, delete the project folder from disk. Default False.

Returns the project's context_doc, recent session history, and next step
formatted as a structured brief. Use this at the start of any work session
on a specific project so Claude has full background.

Args:
    project_id: The project slug (e.g. "hat-dashboard", "article-1").

Call this at the end of any work session on a project. The history feeds
into load_project_context() so future sessions automatically know what
happened before.

Args:
    project_id: The project slug.
    what_was_done: 1-3 sentence summary of what was accomplished this session.
    next_steps: Optional: what needs to happen next. Updates the next_step field.

Useful for onboarding — finds existing project folders that are not yet
tracked in Metis. Call create_project() for each item you want to register.

Args:
    scan_path: Absolute path to scan. Defaults to the parent of METIS_RC_ROOT.

This is the standard way a Metis agent persists its work: it writes the
markdown to outputs/reviews/{agent_slug}/{date}_{task_slug}.md and, by
default, logs the run so the dashboard's Agents tab tracks it. Use it at the
end of any substantive agent task so the result is filed and discoverable.

Args:
    agent_slug: Slug of the agent that produced the review
        (e.g. "epidemiologist", "writing-partner").
    task_slug: Short kebab-case slug identifying the task; becomes part of
        the filename (e.g. "article1-methodology").
    content: The full review content as markdown.
    log_run: Whether to also record this as an agent run for the dashboard.
        Defaults to True.

Returns:
    A confirmation with the path of the saved review file.

Args:
    status: Filter by status -- "open", "done", "blocked", or "" for all.
    project_id: Filter by project. Empty = all projects.
    owner: Filter by owner. Empty = all owners.
    limit: Maximum results (default 25).

Args:
    title: Short task description.
    project_id: Which project this task belongs to.
    owner: Who is responsible (default "Metis").
    notes: Additional details or context.
    due_date: Optional due date in YYYY-MM-DD format.
    recurrence: Optional repeat — "daily", "weekly", "monthly", or "yearly".
                When a recurring task is completed, the next occurrence is created automatically.
    parent_task_id: Optional parent task — set this to make this a subtask.

The companion to create_task and get_tasks: use this to mark a task done or
blocked, reschedule it, reassign it, or edit its details. Only the fields you
pass are changed; empty arguments leave the existing value untouched. Marking
a recurring task "done" automatically creates its next occurrence. Find a
task_id with get_tasks; use delete_task to remove a task entirely.

Args:
    task_id: ID of the task to update (as shown by get_tasks). Required.
    status: New status — "open", "done", or "blocked". Empty = unchanged.
    title: New title. Empty = unchanged.
    owner: New owner. Empty = unchanged.
    notes: New notes/details. Empty = unchanged.
    due_date: New due date in "YYYY-MM-DD" format. Empty = unchanged.
    recurrence: New repeat — "daily", "weekly", "monthly", or "yearly".
        Empty = unchanged; pass "none" to clear an existing recurrence.

Returns:
    A confirmation listing the changed fields (and the next-occurrence id if a
    recurring task was completed), or a note if the task_id was not found.

The destructive complement to create_task — removes the task row entirely.
Use this for tasks created in error or no longer relevant; to instead mark
work finished (and continue a recurring series), use update_task with
status="done". Find the task_id with get_tasks. This cannot be undone.

Args:
    task_id: ID of the task to delete (as shown by get_tasks). Required.

Returns:
    A confirmation that the task was deleted, or a note if no task with that
    id exists.

Use this whenever they ask to be reminded of something, or whenever you agree
to come back to something later. It appears as a chip on the Work calendar on
the chosen day.

Args:
    text: What to remind them of. Write it so it still makes sense in a week
        with no other context — "re-run the routing audit", not "check that".
    date: When. Either YYYY-MM-DD, or a phrase: "tomorrow", "in a week",
        "in 3 days", "in two weeks", "in a month".
    project_id: Optional project slug to file it under (e.g. "metis-dashboard").
        Leave empty for a reminder that belongs to no project — that is a
        normal case, not a fallback.
    remind_at: Optional time of day, "HH:MM".
    until: Optional end date, same formats as `date`. Its meaning depends on
        `repeat`, and the two readings are mutually exclusive:
          * without `repeat` — a MULTI-DAY event. `date`..`until` is one
            continuous block (a conference, a field trip, leave).
          * with `repeat` — when the SERIES stops. Leave empty for open-ended.
        A recurring multi-day event is not supported; say so rather than
        silently picking one of the two readings.
    repeat: "" (default), "daily", "weekdays", "weekly", "monthly" or
        "yearly". A repeating plan is stored as one row plus this rule and
        expanded when the calendar is drawn, so changing it changes every
        future occurrence at once.
    duration_days: How many days ONE occurrence covers, default 1. Use it with
        `repeat` for something like "the first three days of every month".
        Without `repeat`, prefer `until` — it says the same thing more clearly.

Returns:
    Confirmation naming the date, span or schedule it was placed on.

Auto-extracts tags from content. Links to domains/projects if keywords match.
Unless `auto_cross_pollinate=False`, automatically surfaces up to 5 cross-
pollination matches from library / meetings / news / older ideas in the same
response — so the user sees connections without a second tool call.

Args:
    content: The idea text.
    source: Where the idea came from (default "manual").
    image_path: Optional path to an associated image.
    auto_cross_pollinate: When True (default), include connection matches in the response.

Use this to review what you've been thinking about over a chosen time
window — the ideas you logged with capture_idea — so you can revisit,
connect, or act on them. Pairs with capture_idea (to add) and
cross_pollinate (to surface related work).

Args:
    scope: Time window to retrieve. One of "today", "week" (last 7 days),
        "month" (last 30 days), or "all". Defaults to "week".
    limit: Maximum number of ideas to return, newest first. Defaults to 20.

Returns:
    A formatted list of matching ideas with their timestamps and tags, or a
    friendly note if none were found in that window.

Args:
    content: The journal entry text.
    image_path: Optional path to an associated image.

Unlike add_journal_entry (which creates a new row each time), daily_note
keeps a single entry per calendar day and appends timestamped lines to it —
the Reflect/Tana "daily note" pattern for fast, low-friction capture.

Args:
    text: Line to append. Leave empty to just read today's note so far.

Use this to look back over your dated journal/log entries — reflections,
progress notes, and session handoffs — optionally from a given start date.
Helpful for "what was I working on lately?" and for rebuilding context at
the start of a session.

Args:
    date_from: Earliest entry date to include, as "YYYY-MM-DD". Empty
        string (the default) applies no date filter and returns the most
        recent entries.
    limit: Maximum number of entries to return, newest first. Defaults to 10.

Returns:
    A formatted list of journal entries with their dates, or a note if none
    match.

Args:
    name: Contact's full name (used as unique key).
    notes: Notes about this contact.
    role: Contact's role or affiliation.
    birthday: Birthday in YYYY-MM-DD format.

Maintains a personal glossary of field-specific terms and acronyms so
Metis can give consistent definitions across sessions. Upserts on the
term (an existing term keeps its created_at but takes the new definition).
Retrieve entries with get_glossary.

Args:
    term: The term, acronym, or phrase to define; serves as the unique key,
        so reusing an existing term overwrites its definition.
    definition: The definition text to store for this term.

Returns:
    A confirmation message naming the term that was added or updated.

Searches library_seeded, meetings, and news_briefs tables for related
content using keyword matching.

Args:
    content: Text snippet to find connections for.
    limit: Maximum results per source (default 5).

Searches library_seeded, meetings, news_briefs, and ideas tables
for related items. Returns top 5 with source type, title, and snippet.

Args:
    content: Text to find cross-domain connections for.

Gathers recent content from selected sources, respecting an 8000 char
limit per source. Returns assembled context string with source labels.

Args:
    sources: List of sources to include: "library", "meetings", "news", "ideas", "journal".
    date_filters: Optional dict with source-specific date filters, e.g. {"ideas": "2026-03-01"}.

Call this as the very first step whenever the researcher pastes or shares
rows about individual people (a line list, patient or participant records,
a table with names, dates of birth, phone numbers, GPS points or IDs) —
before any calculation or answer, even for a simple question. If the result
is not PUBLIC, do not analyse the raw rows: mask them with anonymize_text,
log the event with log_consent_event, and offer a script the researcher
runs locally.

Returns safety status, classification, specific warnings and the next steps.

Args:
    content: Text content to scan.
    file_path: Optional file path for context-based classification.

Reads tracked_files table, checks actual file modification times,
and updates last_scanned timestamps.

Registers one file so Metis notices when it changes and can read it later
via read_file; tracked files surface on the dashboard's Planning tab. The
file's current modification time is recorded and watch is set on. Re-adding
an existing path updates it (and keeps the old label unless a new one is
given). To register a whole project at once, use connect_project_folder.

Args:
    path: Absolute path to the file to track; the file must exist or an
        error is returned.
    label: Optional category label for the file (default empty string);
        on re-add, an empty label leaves the existing label unchanged.

Returns:
    A confirmation message naming the tracked file (and its label, if any),
    or a "file not found" / error message.

Walks the folder recursively and adds every file with a recognised extension
(.R, .Rmd, .md, .py, .js, .ts, .sql, .json, .yaml, .qmd, .tex, .csv) to
the tracked_files table. Call this once per project; after that, use
read_file() to read any individual file.

Args:
    folder_path: Absolute path to the project root folder.
    label: Short label for all files from this project (e.g. "MLM Course").
    max_files: Safety limit — stop after registering this many files (default 200).

Works for any text file: R scripts, markdown, Python, JSON, CSV, etc.
The file does not need to be pre-registered in tracked_files.

Args:
    path: Absolute path to the file to read.
    max_chars: Maximum characters to return (default 8000). For large files,
               increase this or ask for a specific section.

Args:
    folder_path: Absolute path to the folder.
    pattern: Glob pattern to filter files (e.g. "*.R", "*.md"). Default: all files.

Refuses to touch basket/private/ items.

Args:
    source_path: Absolute path to the file in basket/ to promote.
    target_path: Absolute destination path (file or folder).

Removes a single file from the tracked-files list (the files the dashboard
Planning tab scans for activity). Use this when a file is no longer relevant
or was added by mistake. The file on disk is never touched — only its
tracking record is deleted. The inverse of add_tracked_file.

Args:
    path: Absolute path of the file to stop tracking. Must match the path
        exactly as it was registered.

Returns:
    A confirmation that tracking stopped, or a note if the path was not in
    the tracking list.

The session half of the daily-brief round-trip, for machines whose dashboard
cannot call the API itself. Returns the SAME instructions and the SAME
context the dashboard's own generator uses (metis_mcp.tools.brief_prompt and
assemble_*_context), so a brief written here reads like one written there:
continuous prose, the freshness contract, and a coverage footer that keeps
tomorrow's brief from repeating today's lead. Write the brief by following
the instructions exactly, then pass it, footer included, to save_daily_brief
with the date key this returns.

Args:
    period: "daily" (default), "weekly", or "catchup".

Returns:
    The writing instructions, the context, and the exact save call to make.

The write-back half of the brief round-trip: Claude Desktop or Claude Code
writes the brief from generate_daily_insight(), then calls this. It stores
the brief exactly as the dashboard's own generator would — the coverage
footer is stripped and recorded (so tomorrow's brief does not repeat
today's lead), and the source links shown under the brief are filled in.

Args:
    content: The finished brief, ending with its <<THREADS: ...>> line. Required.
    sources: Leave empty — the source links are filled in from the news table.
    date: The key generate_daily_insight returned: YYYY-MM-DD, or with a
        -weekly / -catchup suffix. Empty = today's daily brief.
    model: Provenance tag. Empty = desktop-brief / desktop-weekly /
        desktop-catchup, matching the suffix, which is what the dashboard reads.

Returns:
    Confirmation with the key saved and what was recorded.

This is what stops the same long-running story leading every morning. Metis
only puts a thread on cooldown once the brief that carried it was actually
marked read — a brief that was generated but never read delivered nothing, so
it must not silence anything. Call this when you have delivered a briefing to
the researcher in conversation, or when they say they have read it.

Args:
    date: YYYY-MM-DD of the brief. Empty = today.
    period: "daily", "weekly" or "catchup". Default "daily".

Returns:
    Confirmation, plus which threads that brief had covered.

Answers "why didn't today's brief mention X?" and "what has Metis been
holding back?". Story threads group many news items into one running story
(an epidemic, a funding shift), so a long-running event can be surfaced once
and then held quiet instead of leading every single morning.

Args:
    days: Window of news activity to consider. Default 7.
    show_all: Include threads with only one item. Default False (recurring only).

Returns:
    Per thread: size, age, whether it may lead, how long it stays quiet, and
    which analytical angles have already been used on it.

"Missed" means never delivered — not merely old. A brief that was generated
but never marked read delivered nothing, so its stories still count as
missed. This is what a catch-up briefing should be built from.

Args:
    days: How far back to look. Default 14.
    min_signal: Minimum signal strength — "high", "medium" or "low". Default "medium".

Returns:
    Story threads with recent activity that no read briefing ever carried.

Args:
    date: Date in YYYY-MM-DD format. Empty = today.

Args:
    topic: Filter by topic tag. Empty = all topics.
    limit: Maximum results (default 20).
    unread_only: If True, only return unread publications (default True).

Clears items from the "new publications" queue once the user has seen them,
stamping each with a read time so they stop resurfacing. Use the IDs
returned by get_new_publications.

Args:
    ids: List of new_publications row IDs to mark as read; an empty list is
        a no-op.

Returns:
    A confirmation message with the count of publications marked as read.

Args:
    topic: Topic name (unique).
    description: Optional description of what to look for.

Sets status='archived' in library_seeded table. Item stays available
for search and cross-pollination but disappears from default view.

Args:
    relative_path: The relative_path primary key in library_seeded table.

De-indexes a paper or document by deleting its row from library_seeded. By
default the file on disk is left untouched (index-only removal); set
delete_file=True to also delete the file, which is guarded so only paths
inside the PKM root can be removed. To hide rather than remove an item, use
archive_library_item instead.

Args:
    relative_path: The relative_path primary key identifying the row in the
        library_seeded table.
    delete_file: If True, also delete the underlying file from disk (subject
        to the within-PKM-root safety check); if False (default), only the
        index row is removed.

Returns:
    A confirmation message of what was removed, or a not-found / error
    message if the item or table is missing.

Searches library_seeded table across basename, relevance_note, disease,
geography, method fields. By default excludes archived items.

Args:
    query: Search term.
    include_archived: Include items with status='archived'. Default False.
    limit: Max results. Default 20.

Writes one Markdown note per library item (YAML frontmatter + abstract) plus
an index, with [[wikilinks]] between items that share a tag — the
claude-obsidian / LLM-Wiki pattern. Gives you a portable, human-readable,
git-diffable view of the knowledge graph you can open in Obsidian. Read-only
on the database; writes only Markdown.

Args:
    out_dir: Destination folder. Default: outputs/knowledge-export/ under the RC root.

Returns the complete YAML content (research interests, data sensitivity,
specialist contexts, etc.). For a lightweight profile summary (name,
interests, news_topics), use get_user_profile() instead.
Creates the config file with defaults if it does not exist yet.

Specialist contexts tell Metis which domains the user works in so routing
and search can be tailored. This appends to specialist_contexts in
user-config.yaml (creating the file with defaults if absent); if a context
with the same name already exists, its description is updated instead of
duplicated. Related tools: toggle_context, list_contexts.

Args:
    name: Short label for the context, e.g. "Epidemiological dashboards";
        also the key used to detect and update an existing context.
    description: One or two sentences describing what this context covers.
    active_by_default: If True (default), the context is added to
        active_contexts immediately; if False, it is stored but left
        inactive (and removed from active_contexts if already present).

Returns:
    A confirmation message stating whether the context was added (and
    activated) or an existing one was updated.

Args:
    name: Context name to toggle (must exist in specialist_contexts).
    active: True to activate, False to deactivate.

Merges the provided YAML into the existing config so that specialist_contexts
and active_contexts set by earlier tools are preserved.

Args:
    yaml_content: Complete YAML string as produced by the wizard (all sections).

Merges the provided JSON into any existing preferences so incremental wizard
saves do not overwrite earlier sections.

Args:
    json_content: JSON string with preference keys (news_topics, journals,
                  pubmed_query, openalex_query, theme, density, etc.).

Reads the file, splits it into discrete items (paragraphs, bullet points,
or numbered items), and calls capture_idea() for each non-empty item.
Supports .txt, .md, and .docx files.

Args:
    file_path: Absolute or METIS_RC_ROOT-relative path to the ideas file.

Saves to {pkm_root}/outputs/images/YYYY-MM-DD_[slug].png

Args:
    prompt: Description of the image to generate.
    backend: "gemini" (default) or "huggingface"
    model: "flash" (gemini-3.1-flash-preview-image-generation),
           "imagen" (imagen-4.0-generate-001),
           "flux" (FLUX.1-schnell via HuggingFace)
    width: Image width in pixels (default 1024).
    height: Image height in pixels (default 1024).
    output_filename: Optional custom filename (without extension).

Reads from {pkm_root}/outputs/images/ directory.
Returns filename, date, and prompt (from JSON sidecar if present).

Args:
    limit: Maximum number of images to return (default 20, newest first).

Each event is a small piece of evidence — you acted on a brainstorm, rated
an idea highly, flagged an agent's output — that feeds your evolving
"thinking profile". Over time these signals let Metis tailor its routing,
suggestions, and tone to your preferences. Call it whenever a meaningful
preference moment occurs; read the accumulated profile with
get_thinking_profile.

Args:
    event_type: The kind of signal. Must be one of: "brainstorm_acted_on",
        "brainstorm_ignored", "idea_rated_high", "idea_linked_project",
        "journal_revisited", "agent_output_accepted", "agent_output_flagged".
    source_type: Domain or category the signal belongs to (e.g. "biology").
        Optional; defaults to empty.
    content_id: ID of the related content record (idea, journal entry, etc.)
        if applicable. Optional; defaults to 0 (none).
    agent_slug: Agent identifier this signal relates to, used for the
        "agent_output_*" event types. Optional.
    context: Free-text note giving context for the event. Optional.

Returns:
    A confirmation that the signal was recorded, or an error listing the
    valid event types if an invalid one was supplied.

Computes:
- connection_preferences: acted-on rates per domain_pair (source_type)
- preferred_idea_sources: frequency of source_type in high-rated idea events
- agent_feedback: accepted/flagged rates per agent_slug

Writes updated system/thinking-profile.yaml. Safe to call multiple times.

This erases all recorded preference signals and restores the default profile.

The proposal is queued for human review. The skill file is NOT modified
until the user calls approve_proposal().

Args:
    agent_slug: The agent's slug (e.g. 'librarian', 'writing-partner')
    proposed_content: The full proposed replacement content of the skill file
    rationale: Why this change is being proposed (1–3 sentences)

Returns:
    Confirmation with the proposal ID for the user to review

Writes the proposed content to the agent's skill.md file and marks
the proposal as approved. Creates a backup of the current skill file first.

Args:
    proposal_id: The numeric ID from get_pending_proposals()

The skill file is not changed. The proposal is marked rejected with an
optional reason.

Args:
    proposal_id: The numeric ID from get_pending_proposals()
    reason: Optional note explaining why the proposal was rejected

Called at the end of every agent run to capture experience: what worked,
what could be better, what context was missing, what tools were needed.
Entries are reviewed by the weekly Coach loop for self-improvement proposals.

Args:
    session_id: Pipeline session ID from session_bootstrap().
    agent_slug: Which agent is writing the reflexion (e.g. 'librarian').
    went_well: What went well in this run (1–2 sentences).
    could_improve: What could have been done better (1–2 sentences).
    missing_context: What context or data was unavailable but needed.
    tool_wishes: Tools or capabilities that would have helped.

Optionally applies speaker diarization with pyannote.audio if installed
and HF_TOKEN is set. Saves the transcript alongside the audio file and
updates the meetings table.

Args:
    recording_id: The meeting_id from the meetings table (shown in Meetings tab)

Returns:
    Transcript text (with speaker labels if diarization succeeded) and
    the path where it was saved.

Looks across Metis's long-term memory to recall past context — what was
decided, found, or noted before. It searches the memory_entries table
(title, summary, topics) and also greps journal/**/*.md files on disk, so
both structured memory and freeform journal notes are covered.

Args:
    query: Keyword or phrase to match against entry titles, summaries,
        topics, and journal note text.
    entry_type: Optional filter limiting results to one kind of entry —
        "session", "journal", "idea", "decision", or "topic". Empty string
        (default) searches all types.

Returns:
    A text block of matching memory entries and journal hits, or a message
    when nothing matches.

Use this for a human-curated 'memory palace' note (title + summary + topics,
optionally saved as a markdown file). For machine/agent event logging use
store_episodic_memory; for a distilled concept/definition use
store_semantic_memory.

Inserts into the memory_entries table. If detail is provided, also writes
a markdown file under journal/{entry_type}s/.

Args:
    title: Short title for the entry.
    summary: One-paragraph summary, stored in the DB and shown in search.
    topics: Comma-separated topic tags, e.g. "metis-setup,mcp-server".
    entry_type: One of "session", "journal", "idea", "decision", or "topic".
    detail: Full markdown content for the optional .md file.
    computer: Hostname of the computer this entry is from (optional).

Returns:
    A single TextContent confirming the saved entry (title, generated ID,
    type, topics, and the markdown file path if one was written), or an
    error message if the database is missing or the write fails.

Args:
    topic: Topic tag to filter by (e.g. "metis-setup", "phd-research").

Useful at the start of a session to recall what was last worked on.

Args:
    n: Number of entries to return (default 10).

Checks for an active session (same computer, last active within 2 hours).
If found: resumes it and returns the last 5 events.
If not: creates a new session and seeds context from recent memory.

Args:
    client: Which Claude client is calling ('code'|'chat'|'cowork'|'dashboard').

scope='always' writes a permanent, high-priority rule (beats the built-in seeds)
into the routing DB — part of institutional memory, persists across Metis updates.
scope='once' is acknowledged but not stored.

Args:
    phrase: the trigger word/phrase the user wants routed (e.g. "spatial scan").
    agent_slug: which agent to route it to (e.g. "epidemiologist").
    scope: 'always' (persist) or 'once' (don't store).
    priority: lower = more specific / checked first (default 5 beats all seeds).

Use this whenever the user states (or confirms) a standing preference or makes a
decision worth remembering — coding style, citation format, a methods default, an
article/dataset they keep returning to, a naming convention, a workflow choice.
These are recalled into context on future requests (recall_decisions), so Metis
personalizes instead of asking again.

Args:
    decision: the preference/decision in plain language (e.g. "Always use tidyverse style in R, never base apply").
    category: preference | coding | citation | methodology | writing | article-ref | dataset | routing | other.
    context: optional — when/why it applies (e.g. "for HAT spatial analyses").
    scope: 'always' (persist) or 'once'.
    agent_slug: the specialist that should APPLY this decision — e.g.
        "frontend-designer-builder" for a layout rule, "writing-partner" for
        a prose rule, "librarian" for what matters in the library. Leave
        empty for a project-wide rule that every agent should carry.

        Attribution is what makes the decision reachable. `get_agent_context`
        injects an agent's own decisions plus the project-wide ones, and a
        flat list of every preference injected into every agent is noise that
        gets ignored. Measured 2026-08-24: user_decisions held 2 rows while
        session summaries held 7,578 decision entries — so invoking a
        specialist returned a persona and nothing about how the researcher
        wants things done, which is why routing to one stopped being worth it.

Args:
    category: filter to one category, or '' for all.
    limit: max rows.

Returns a verdict (OK / REVIEW), the preferences this answer must honor, and any
detected conflicts (e.g. a 'never use base apply' preference when the answer shows
`apply(`). Resolve REVIEW items before replying.

Args:
    answer: your drafted answer text.
    session_id: current session (optional).
    task_type: optional routing task_type for narrower preference recall.

Call this after every tool call, file write, and classification decision.
Event types: 'turn' | 'tool_call' | 'result' | 'file_write' | 'redline' | 'classification'

Args:
    session_id: Session ID from session_bootstrap().
    event_type: Category of event being recorded.
    content: Event content (truncated to 2000 chars).

Every /metis invocation passes through here. The pipeline:
  1. Bootstraps or resumes the session
  2. Classifies content (PUBLIC/INTERNAL/CONFIDENTIAL/SENSITIVE)
  3. Data Guardian: blocks SENSITIVE requests outright
  4. Cybersecurity: blocks prompt injection and suspicious URLs
  5. Parses intent and selects the appropriate agent(s)
  6. Allocates model and token budget
  7. Assembles minimum surgical context from memory
  8. Persists the turn to session_events
  9. Returns routing decision — agents execute and then call:
       save_session_event(..., 'result', output)
       log_agent_run(..., session_id=session_id)
       write_reflexion(session_id, agent_slug, ...)

Stages 10 (logging) and 11 (reflexion) are called by the executing agent
after completing their work.

Args:
    request: The researcher's request text.
    session_id: Existing session ID if resuming. Leave empty to auto-bootstrap.
    client: Which Claude client is calling ('code'|'chat'|'cowork'|'dashboard').
    max_turns: Maximum pipeline turns before graceful truncation (default 20).

Logs a time-stamped EVENT (something that happened) and indexes it for vector
search. For a distilled, timeless concept/definition use store_semantic_memory;
for a human-curated palace note use add_memory_entry.

Episodic memory is a chronological log of things that happened — ideas,
notes, papers read, tasks completed, agent runs.

Args:
    content: The text content of the event to remember.
    event_type: One of 'idea', 'note', 'task', 'paper', 'meeting', or
        'agent_run'.
    session_id: Current pipeline session ID (optional).
    metadata: JSON string with extra fields such as title, tags, or source.

Returns:
    A single TextContent confirming the stored event (its row id and type),
    or an error message if the database is missing, fastembed is not
    installed, or the write fails.

Stores a distilled CONCEPT/definition (timeless 'what I know'). For a
time-stamped event use store_episodic_memory; for a human-curated palace
note use add_memory_entry.

Semantic memory holds the 'what I know' layer — concepts, definitions,
and their relationships. Used by retrieval to surface relevant knowledge
without relying on raw event history.

Args:
    concept: Short name for the concept, e.g. 'RDT sensitivity' or
        'fAChE inhibition'.
    definition: A one-to-three-sentence definition or explanation.
    related_concepts: Comma-separated names of related concepts.
    source_type: Where this came from: 'paper', 'note', 'idea', or
        'user_defined'.
    source_id: ID of the source record, e.g. a paper DOI or idea_id.

Returns:
    A single TextContent confirming the stored node (its row id and concept
    name), or an error message if the database is missing, fastembed is not
    installed, or the write fails.

Procedural memory captures 'how to do things' — repeatable processes,
workflows that worked well, or step-by-step patterns for recurring tasks.

Args:
    procedure_name: Short name for this procedure (e.g. 'Domain literature search').
    steps: Markdown-formatted steps for the procedure.
    trigger_context: What situation should trigger using this procedure.

Working memory is an ephemeral scratchpad — it persists for the session
but is not indexed for vector search. Use it for state that agents need
mid-pipeline (e.g. intermediate results, decisions made so far).

Args:
    session_id: Pipeline session ID from session_bootstrap().
    key: Variable name (e.g. 'current_article', 'user_intent').
    value: Value to store (any string, JSON, or text).

Args:
    session_id: Pipeline session ID. Leave empty to get the 20 most recent entries
                across all sessions.

Scans each requested layer for rows without a corresponding vector in the
vec_* table, generates embeddings, and inserts them. Safe to run repeatedly
— only processes gaps, never overwrites existing vectors.

Use this to recover from embedding failures, or after importing memories
via raw SQL. Can be scheduled nightly via APScheduler.

Args:
    layers: Comma-separated layers to reindex: 'episodic', 'semantic',
        'procedural' (default: all three).
    batch_size: How many entries to embed per batch (default 50).
        Lower values use less memory; higher values are faster.

Returns:
    Summary of how many entries were reindexed per layer.

Searches your personal MEMORY layers (episodic/semantic/procedural — things
you or Metis recorded), NOT your document library. For documents/PDFs use
search_pdf_knowledge; for reference metadata use search_library.

Retrieval pipeline (M5.8):
1. Embed the query.
2. Run vector similarity search on each requested layer.
3. Run keyword LIKE search on each layer.
4. Fuse results using Reciprocal Rank Fusion (RRF, k=60).
5. Return top_k deduplicated results ranked by fused score.

Args:
    query: Natural language search query to embed and keyword-match.
    layers: Comma-separated memory layers to search, drawn from 'episodic',
        'semantic', and 'procedural'.
    top_k: Number of fused results to return (default 5).

Returns:
    A single TextContent listing the top fused results (layer, title, score,
    timestamp, and a content preview), or a "no results" / error message.

Call this before inserting any externally-sourced content into agent context:
web scrapes, RSS items, PDF extracts, YouTube transcripts, GitHub readmes.

Args:
    content: The external content to probe.
    source_label: Human-readable label for logging (e.g., "PubMed abstract", "RSS item").

Returns:
    JSON with {probed_content, flagged, patterns_found}.

Returns a compact summary of behavioral rules appropriate for the
complexity level. Prepend to any agent's system context to enforce
shared policy across all agent types.

Args:
    level: 'quick' | 'standard' | 'deep' | 'chain'

Treats sub-agent output with the same suspicion as external tool output.
Rejects if any required keys are missing or empty.

Args:
    output_json: JSON string of the sub-agent output dict.
    required_keys: Comma-separated list of required keys (e.g., "title,summary,agent_slug").

Configure a vault via METIS_OBSIDIAN_VAULT env var or `obsidian_vault:` in
user-config.yaml. Re-running is safe (REPLACE semantics).

Args:
    from_path: Relative path from knowledge/library/ (e.g. 'concepts/elimination-framework.md')
    to_path:   Target note path.
    max_hops:  Maximum path length (default 4).

Returns all paths found up to max_hops, ranked by length.

Performs BFS flood-fill from the given note up to `depth` hops,
returning all reachable notes grouped by distance. Useful for surfacing
related concepts when working on a specific topic.

Args:
    note_path: Relative path from knowledge/library/ (e.g. 'disease-areas/[condition].md')
    depth:     Maximum hop distance to explore (default 2).

Scans memory_entries, episodic_memory, semantic_memory, and ideas,
extracts their topic tags, and creates bidirectional edges between
items that share topics. This makes cross-pollination between memory
layers discoverable via kg_memory_connections().

Takes no arguments. Safe to re-run (uses REPLACE semantics).

Returns:
    Summary of nodes and edges indexed.

Uses the memory_links graph built by kg_index_memory(). BFS flood-fill
from the given entry up to `depth` hops.

Args:
    entry_type: Type of the starting entry: 'memory', 'episodic', 'semantic', or 'idea'.
    entry_id: ID of the entry (entry_id for memory, numeric id for others).
    depth: Maximum hop distance (default 2).

Returns:
    Connected entries grouped by distance, with their shared topics.

Args:
    name:       Human-readable span label (e.g. 'stage_1_bootstrap', 'tool:search_library').
    kind:       Span type — 'internal' | 'tool' | 'agent' | 'llm'. Default: 'internal'.
    session_id: Session identifier (from session_bootstrap). Optional.
    run_id:     FK to agent_runs.run_id. Optional.
    parent_id:  Parent span_id for nested spans. Optional.
    tags:       JSON string of extra key/value metadata. Optional.

Returns the span_id string — pass it to end_span() when the work is done.

Args:
    span_id: The span_id returned by start_span().
    status:  'ok' | 'error'. Default: 'ok'.
    error:   Error message if status='error'. Optional.

Returns a summary line: '{name} — {duration_ms}ms [{status}]'

Useful for logging retrospective timing data (e.g. 'that DB query took 42ms').

Args:
    name:        Span label.
    duration_ms: How long the work took in milliseconds.
    kind:        'internal' | 'tool' | 'agent' | 'llm'. Default: 'internal'.
    session_id:  Session identifier. Optional.
    run_id:      FK to agent_runs. Optional.
    parent_id:   Parent span_id. Optional.
    status:      'ok' | 'error'. Default: 'ok'.
    tags:        JSON string of metadata. Optional.

Returns the new span_id.

Args:
    session_id: Filter to this session. Optional.
    run_id:     Filter to this agent_runs.run_id. Optional.
    limit:      Max rows to return. Default: 50.

Returns a JSON array of span objects.

Replaces:
  - Patient/case IDs        → [PARTICIPANT_001]
  - GPS coordinates         → [GPS_001]
  - Belgian national IDs    → [NID_001]
  - Email addresses         → [EMAIL_001]
  - Phone numbers           → [PHONE_001]
  - Name-like tokens (opt.) → [NAME_001]

Args:
    content:        Text to anonymize.
    mode:           'full' — replace; 'preview' — mark without replacing.
    replace_names:  Also replace CAPITALIZED name-like tokens (heuristic).

Returns JSON with keys 'anonymized' (str) and 'replacements' (dict).

The output-side complement to the read-side hook. Agents call this on any
response that might contain individual-level data; it enforces the
constitution's no-pii-output rule. Returns a verdict, what was found, and a
masked version safe to send.

Args:
    text: The drafted response text to check.

Returns JSON: {safe: bool, found: {type: count}, masked: str}.

The redaction half of /safe-analysis: where check_data_safety only *detects*
sensitive data and the read-hook *asks*, this returns a masked version so an
approved read shares no raw identifiers. Sensitive columns (patient/case IDs,
names, GPS, etc.) are pseudonymised consistently — the same value always maps
to the same placeholder, so record linkage survives while identity does not.
PII patterns (emails, phones, IDs) are scrubbed from all remaining cells.

Local I/O only — nothing leaves the machine except the masked preview you see.

Args:
    path:     Absolute local path to a CSV/TSV/text data file.
    max_rows: Rows to include in the masked preview (default 20).

Returns JSON: redacted preview rows, the columns masked, and a per-type count.

Args:
    original:   Original (pre-anonymization) text.
    anonymized: Anonymized text from anonymize_text().

Returns a plain unified-diff string suitable for display.

Call this whenever an agent processes data so there is an audit trail
of what was processed, when, and under what classification.

Args:
    action:              Short description — e.g. 'scan_document', 'anonymize_patient_data'.
    data_classification: 'PUBLIC' | 'INTERNAL' | 'CONFIDENTIAL' | 'SENSITIVE'.
    agent_slug:          Which agent performed the action.
    notes:               Free-text context.
    session_id:          Current session identifier.

Returns the data-handling consent trail — what data was approved or
classified, by which agent, and when — so you can review or report on how
sensitive data has been treated. Reads the consent_ledger that
log_consent_event writes to.

Args:
    limit: Number of most recent ledger rows to return, newest first
        (default 30).

Returns:
    A JSON text block listing the consent events (id, timestamp, action,
    data_classification, agent_slug, notes).

'strict'  — No internet access. Only local DB, files, and MCP tools.
'normal'  — Default. Librarian and News Radar may access allowed domains.
'offline' — Airplane mode. All external requests blocked.

Args:
    policy: One of 'strict' | 'normal' | 'offline'.

Uses SQLite's Online Backup API — safe to run while the database is live.

Args:
    destination:       Directory path to write the backup into.
                       Defaults to metis/system/backups/.
    label:             Optional label appended to the filename, e.g. 'pre-upgrade'.
    extra_destination: Optional second directory for off-site / secondary copy.

Returns JSON with backup_path, size_kb, checksum (SHA-256), and elapsed_ms.

Produces <backup_path>.enc. The passphrase is never stored.
Uses Python stdlib only (hashlib for key derivation, os.urandom for salt/nonce).

NOTE: This uses a simple PBKDF2+AES-GCM implementation.
For production-grade encryption, use a proper secrets manager.

Args:
    backup_path: Full path to the .sqlite backup file.
    passphrase:  Encryption passphrase.

Returns JSON with enc_path and whether original was removed.

Reverses the format [8B magic][16B salt][12B nonce][ciphertext+tag]. Writes the
recovered .sqlite next to the .enc file (dropping the .enc suffix) unless
output_path is given. The passphrase is never stored. This is the counterpart
to encrypt_backup — without it, encrypted backups could not be restored.

Args:
    enc_path:    Path to the .enc encrypted backup.
    passphrase:  The passphrase used at encryption time.
    output_path: Optional explicit destination for the decrypted file.

Returns JSON with out_path and the method used.

Args:
    backup_dir: Directory to scan. Defaults to metis/system/backups/.

Returns JSON array, newest first.

Args:
    backup_path: Full path to an unencrypted .sqlite backup.

Returns JSON with status ('ok' or errors) and table count.

Renames the current live database to <db>.pre-restore.<timestamp>
before overwriting, so you can recover if something goes wrong.

IMPORTANT: This overwrites the live database. All changes since the backup
was taken will be lost. The dashboard must be restarted after restore.

Args:
    backup_path: Full path to the .sqlite backup to restore from.
    confirm:     Must be the string 'YES' to proceed.

Returns JSON with status and paths.

The schedule is read by the scheduler (Phase 10) to trigger backup_db()
automatically. This tool only persists the configuration.

Args:
    enabled:     Whether automatic nightly backups are on.
    time_utc:    Time in HH:MM UTC to run the backup (e.g. '02:00').
    keep_days:   How many days of backups to retain (older ones deleted).
    destination: Backup directory (defaults to metis/system/backups/).

Call this at the start of a brainstorm and after each steering action.
The tool fetches relevant ideas, notes, questions, and library notes so
you (the AI) can surface connections the user may not have considered.

Steering modes:
  expand    — broaden; surface tangential connections
  focus     — narrow; find the most relevant threads
  challenge — find counter-arguments and weaknesses
  synthesize — identify themes; propose a unifying framework
  connect   — explicit cross-domain connections to library/literature

Args:
    topic:        The brainstorm topic or question.
    steering:     One of expand|focus|challenge|synthesize|connect.
    session_uuid: Pass the UUID from the previous turn to continue a session.
                  Omit to start a new session.
    turn_notes:   Optional free-text notes from the previous turn to log.

Returns JSON with: session_uuid, turn_number, context (ideas/notes/
questions/library), steering_prompts, and instructions for the AI.

Args:
    session_uuid: The session identifier returned by brainstorm_turn().

Args:
    limit: Maximum sessions to return (default 20).

Writes to outputs/brainstorms/<date>_<slug>.md and updates the
brainstorm_sessions table status to 'saved'.

Args:
    session_uuid: The session identifier.
    title:        Short descriptive title for the brainstorm.
    synthesis:    The key insights and connections (Markdown-formatted).
    action_items: Bullet-point action items or follow-up questions.

Supports CSV, TSV, Excel (.xlsx/.xls), SPSS (.sav), Stata (.dta).
Performs PII column name scan before profiling (non-blocking, annotated).
Never modifies the source file.

Args:
    path:        Absolute local path to the dataset file.
    sample_rows: If > 0, include this many rows as a data sample in the output.

Returns JSON with: path, rows, columns, null stats, duplicate count,
per-column profile (dtype, nulls, distributions or top values),
and any flagged PII column names.

Analyses the profile and suggests operations with rationale, e.g.:
  - "col 'age' has 12% nulls → consider fill_na or drop_na_rows"
  - "7 duplicate rows detected → apply drop_duplicates"
  - "col 'name ' has leading/trailing whitespace → apply strip_whitespace"

Args:
    path: Absolute local path to the dataset file.

Returns JSON with profile summary and a list of suggested operations,
each with: operation, column (if applicable), rationale, priority (high/medium/low).

NEVER modifies the original file. Always writes to output_path.

⚠️ Writing a dataset requires authorization. This tool refuses to write
unless authorized=True (confirm with the user first) or the env var
METIS_ALLOW_DATA_WRITE=1 is set. This mirrors the Claude Code write-gate so
a rebuild can't bypass it via MCP.

Supported operations:
  - "drop_duplicates"                    — remove exact duplicate rows
  - "drop_columns:[col1:col2:...]"       — remove specified columns
  - "fill_na:[col:value]"                — fill nulls in col with value
  - "rename_column:[old_name:new_name]"  — rename a column
  - "strip_whitespace"                   — strip leading/trailing spaces from all string columns
  - "standardize_dates:[col:format]"     — parse col as date (format: 'auto' or strftime)
  - "drop_na_rows:[col]"                 — drop rows where col is null
  - "drop_na_rows_any"                   — drop rows with ANY null value

Args:
    path:         Absolute local path to the source dataset.
    operations:   List of operation strings (see above).
    output_path:  Where to write the cleaned file. If empty, appends '_cleaned'
                  before the extension (e.g. data.csv → data_cleaned.csv).

Returns JSON with: output_path, original_shape, cleaned_shape, row_delta,
col_delta, operations_applied, operations_skipped.

Pass the JSON strings returned by profile_dataset() for the original
and cleaned files. Returns rows added/removed, columns added/removed,
null count changes per column, type changes, and a human-readable summary.

Args:
    before_profile: JSON string from profile_dataset() on the original file.
    after_profile:  JSON string from profile_dataset() on the cleaned file.

Returns JSON with: row_delta, col_delta, column_diffs (nulls, dtypes),
duplicate_delta, and a human_summary string.

Captures: tokens used today, active projects, open tasks, recent agent
runs, and the most recent journal entry. Writes to
`metis/journal/YYYY-MM-DD_session_handoff_auto.md` and returns the brief.

Use this when a session is approaching its end, before `/clear`, or when
switching to another AI / device.

Runs the nightly self-improvement consolidation: every theme an agent raised
>= min_count times in the last `days` becomes a searchable semantic-memory
node (deduped), and working_memory older than 7 days is pruned. Idempotent.

Reads ``reflexion_log`` entries from the last ``days`` days (default 14)
and returns the top recurring 'could improve', 'missing context', and
'tool wishes' themes, ordered by busiest agent.

Pass ``agent_slug`` to limit the scope; leave blank for all agents.

Reads themed reflexions for ``agent_slug``, appends a 'Self-improvement
notes' section to the agent's current skill.md, and queues the result in
``skill_improvement_proposals`` with status='draft'.

The draft is NOT applied. Use ``apply_proposal(id)`` to write it to disk.

The previous skill.md content is backed up alongside as
``skill.md.bak.<timestamp>`` so a revert is always possible. Updates the
proposal row to status='applied' with the applied_at timestamp and the
backup path.

Args:
    proposal_id: the id from skill_improvement_proposals.

Use this throughout a run to capture what you are learning so it can be
recalled in future sessions without re-reading history.

Args:
    observation_type: One of: discovery, decision, implementation, issue, note.
      - discovery:      something new you found out
      - decision:       a choice made and the reasoning behind it
      - implementation: what was built or changed
      - issue:          a bug, blocker, or failure found
      - note:           a general observation that doesn't fit above
    content:        The observation in 1–3 sentences.
    agent_slug:     Which agent is recording (e.g. 'librarian').
    session_id:     Current pipeline session ID (optional).
    concepts:       Comma-separated concept tags — auto-extracted if blank.
    related_files:  Comma-separated file paths this observation relates to.

Progressive disclosure: returns more detail when budget allows.
  ≤ 500 tokens  → index only   (type + first 12 words per entry)
  ≤ 2000 tokens → preview      (type + first 40 words)
  >  2000 tokens → full         (complete content)

Combines semantic vector search (if fastembed available) with keyword
fallback, filtered to the last `days` days.

Args:
    query:        What you are looking for — natural language.
    budget_tokens: How many tokens you can spend on context (default 2000).
    agent_slug:   Restrict to a specific agent's observations (optional).
    days:         How far back to search (default 90 days).

Use this after web-searching for the researcher's field. The Outbreaks board is
modelled on WHO's Disease Outbreak News: the handful of outbreaks that MATTER
right now, each with a location and a date — not a mirror of any one
publisher's feed. (WHO retired the DON RSS feed; the page is the reference,
and ECDC's Communicable Disease Threats Report covers the gap.) The Events
board holds upcoming scientific congresses/conferences/symposia; the Funding
board holds open or upcoming research funding calls, grants and fellowships.
All three are curated — no RSS feed writes to them — so this tool is how
Claude Desktop (on the user's subscription, no API
rate limit) keeps them current — the dashboard's "Update" buttons open
a chat that calls this tool.

Replaces the previously tool-filled rows on that board; items the user curated or
added by hand are preserved.

EVERY ITEM IS CHECKED AGAINST THE BOARD IT CLAIMS. A congress or a committee
session sent to Outbreaks is filed under Events instead; a call for proposals
goes to Funding; a newsletter, bulletin, quarterly report or declaration
belongs on no board and is recorded as turned away, with the reason, rather
than silently dropped. So the saved count may be lower than the number of
items passed, and some may appear on a different board — the return value
says how many of each.

Args:
    board: "outbreaks", "events" or "funding".
    items: list of objects, each {"title": str, "url": str, "date": str (optional,
        event date or application deadline), "description": str (optional, one short
        sentence)}. Only include real items with a working http(s) URL.

Returns:
    dict: {ok, board, saved, re_routed, not_a_board_item} on success, or
    {ok: False, error} on failure.

Scans journalism only — outbreak surveillance (ECDC threats report, WHO AFRO,
PAHO), science reporting (ScienceDaily, Guardian Science, STAT, Nature news,
The Conversation Africa), humanitarian and disaster feeds (GDACS, ReliefWeb,
OCHA), Africa desks and policy sources. Journal tables-of-contents are NOT
scanned here: the primary literature is the Library's job, via
`scan_library`. Deduplicates by URL, so running it repeatedly is safe.

This is the Library's own scan: journal tables-of-contents and preprint
servers (PLOS NTDs, Parasites & Vectors, Malaria Journal, Lancet ID, CDC EID,
BMC Infectious Diseases, Int J Health Geographics, medRxiv, arXiv q-bio …),
written to `new_publications` for the literature-alert surface.

Separate from `scan_news` on purpose: this builds a scientific background
rather than reporting what happened, so it runs on its own schedule and its
output never appears on the News surface. Deduplicates by URL.

Walks all subdirectories. Uses the parent folder name as a domain tag.
Deduplicates by title so running multiple times is safe.

Detects audio files (.m4a, .mp3, .wav, .ogg, .flac, .aac) and, when
auto_transcribe_audio=True (default), transcribes each one with faster-whisper
and captures the transcript as an idea. The audio file is moved to
inbox/processed/ after successful transcription.

Non-audio files are listed but left for manual review.

Args:
    auto_transcribe_audio: When True (default), automatically transcribe
        audio files found in the inbox. Set to False to just list them.

Args:
    limit: Maximum number of briefs to return (default 10).
    source_type: Filter by type — "news" for RSS items, "article" for scientific papers. Empty = all.
    domain: Filter by domain tag (e.g. "HAT", "AI", "public-health"). Empty = all.
    since: ISO date string — only return briefs created after this date. Empty = all.

Why this exists: feedparser never raises on an HTTP error — it returns an
object with zero entries. So a feed returning 404 or 403 was indistinguishable
from a quiet news day, and `scan_news_feeds` reported "0 new items" either way.
Measured 2026-08-19: 14 of 33 news feeds were HTTP-dead and had been
contributing nothing, silently. That is the real reason the News surface looked
thin — not too few sources, but sources that never answered.

Requests are sequential and spaced, because hammering publishers in parallel
produces 403s that are rate limiting rather than real rejections.

Args:
    kind: "news" (journalism, the News surface), "paper" (journal ToCs and
        preprints, the Library), or "all". Default "news".

Returns:
    Per feed: working (with entry count and newest publication date) or the
    HTTP status that needs fixing.

Runs:
1. News feeds (RSS) — new items added to news_briefs
2. Literature folder — new PDFs registered in literature_metadata
3. Inbox — unprocessed items flagged
4. Tracked files — changed files reported

No LLM calls. Safe to run at any time.

The complement to sync_zotero_library (which only pulls DOWN). Finds
literature_metadata rows that did not come from Zotero (no zotero_key) and
creates them as items in your Zotero library, so papers Metis ingested — e.g.
open-access reports added to a knowledge layer — end up in Zotero too.

Args:
    dry_run: When True (default), report exactly what WOULD be created without
        writing anything. Always review the dry run before a real push.
    limit: Max items to push in one call.

Returns:
    A summary: how many candidates, and either the dry-run preview or the
    create result. Requires a WRITE-enabled ZOTERO_API_KEY.

Performs an incremental sync by default — only fetches items changed since
the last sync. Pass full=True to re-sync everything.

Requires ZOTERO_API_KEY and ZOTERO_USER_ID in metis/system/.env.

Args:
    full: If True, re-sync all items regardless of last sync state.

The offline alternative to sync_zotero_library: reads your local Zotero
database file, so it works with no ZOTERO_API_KEY and no network. Items are
imported into literature_metadata (library_source='zotero-local'), the same
table the Librarian and search_library use.

Args:
    db_path: Path to zotero.sqlite. If empty, Metis searches common locations
             (~/Zotero/zotero.sqlite, and /mnt/c/Users/*/Zotero on WSL).

Searches your saved reference METADATA (title/authors/abstract/tags of papers
in your Zotero-synced + manual library). For meaning-based search inside PDF
body text use search_pdf_knowledge; for exact keyword search of PDF text use
search_fulltext; for an online/external literature lookup use search_literature.

Runs a substring search across the user's indexed references (Zotero-synced
plus manually added) so you can find what they already have before going to
the internet. Matches the query against title, authors, abstract, and tags,
returning the newest papers first. For richer literature workflows see
ask_library, search_literature, and export_citations.

Args:
    query: Search terms matched as a substring against title, authors,
        abstract, and tags.
    limit: Maximum number of papers to return, ordered newest year first
        (default 10).

Returns:
    A formatted text list of matching papers (title, authors, year, journal,
    DOI, abstract snippet), or a "no papers found" message.

Uses abstracts and titles to embed papers, then clusters them with k-means.
Returns a proposed collection structure with suggested names and paper counts.

Args:
    n_clusters: Number of topic clusters. 0 = auto-detect (sqrt of library size).
    min_papers: Minimum papers per cluster to report (default 3).

Use this for Mendeley users: export your library from Mendeley as BibTeX,
then point this tool at the file.

Args:
    bibtex_path: Full path to the .bib file (e.g. from Mendeley export).

Use this to see how big and how current your reference collection is before
searching or citing: it reports the total number of papers, a breakdown by
source (e.g. Zotero, Mendeley, manual) and by item type, the most recently
added references, and — if you sync Zotero — when the library was last
synced. A quick "what's in my library right now?" overview. Takes no
arguments. Pairs with search_library and sync_zotero_library.

Returns:
    A formatted summary: total papers, counts by source and item type, the
    five most recent references, and Zotero sync state if available.

Call this during setup or when switching reference managers.

Args:
    provider: "zotero" or "mendeley". Mendeley uses BibTeX export.
    api_key: Zotero API key (from https://www.zotero.org/settings/keys).
    user_id: Zotero numeric user ID (shown on the same settings page).
    bibtex_path: For Mendeley: full path to exported .bib file.

Closes the "no citation-style export" gap: produces a .bib you can import
into Word (via Zotero/Mendeley), LaTeX/Overleaf, or any reference manager,
and use for cite-while-you-write.

Args:
    query: Optional keyword filter over title/authors/journal. Empty = all.
    tag: Optional tag filter (substring match on the tags field).
    collection: Optional collection-name filter.
    fmt: Output format — currently "bibtex" (RIS available via mine_references).
    limit: Max records to export (default 500).

Writes the file to outputs/exports/ and returns its path + a preview.

For texts a background pack cannot download — anything behind a publisher's
browser check or your institution's subscription. Save them in Zotero with the
browser connector (logged in through your library), then point this at the
collection. No Zotero API key needed; it reads your local Zotero database.

The files are COPIED, so your Zotero library is left untouched. Indexing is not
run here — press Rebuild on the Library surface, or wait for the nightly index.

Args:
    collection: Zotero collection name, exactly as it appears in Zotero.
    database:   Slug of the knowledge layer to add them to (e.g. 'ph-foundations').
    confirm:    True to actually copy. Without it you get the list only.

The install doctor's report: the computer (version, disk, internet), Metis's
own programs (Python, tools, search engine, search model), each AI app it
connects to (Claude Desktop, Claude Code, VS Code, Cursor) and whether
Metis starts from that app's settings, the dashboard and its morning scans,
the scheduled tasks, and the data (database, specialists, profile, updates).
To also repair, use verify_install. Tell the person the result in plain words.

The install doctor — the same check the installers run and schedule daily —
over the list in system/install/essentials.json: computer, Metis's
programs, every AI app it connects to and whether Metis starts from that
app's own settings, the dashboard and its scans, the scheduled tasks, and
the data. With fix=True (default) it reinstalls broken or outdated tools,
reconnects apps (merging, never replacing their other connectors), adds
missing database tables, downloads the search model, starts the dashboard
and re-creates missing scheduled tasks — then checks again.

Use when the person says something is not working, after an update, or when
they ask whether everything is set up. Tell them the result in plain words;
if an app was reconnected, they must quit it completely (Claude Desktop:
also from the tray) and open it again.

For installs made by MetisSetup.exe on Windows. Without confirm it only
reports the installed and the newest published version and what updating
does — ask the person before calling again with confirm=True. Updating
keeps their profile, library, projects and settings.

After confirm=True the installer shows a small progress window and stops
Metis while it works, so this conversation loses Metis for a few minutes.
Tell the person: wait until the progress window closes, then quit Claude
Desktop completely (also from the tray) and open it again.

Fetches references for one or more DOIs (comma-separated) via CrossRef,
checks against your Zotero library, and reports what's missing.

Args:
    dois:  Comma-separated list of DOIs to mine.
    label: Optional label for the output file.

Reads PDFs from inputs/literature/ and Zotero storage.
Stores first 4000 characters per paper in library_fulltext table.
Also updates literature_metadata.abstract for any matched papers.
Incremental — only processes files not yet indexed.

Args:
    scope: "literature" = inputs/literature/ only | "zotero" = Zotero storage only | "all" = both

Exact KEYWORD search across the full body text of your indexed PDFs. For
meaning-based (semantic/vector) PDF search use search_pdf_knowledge; for
reference metadata only use search_library.

Searches the library_fulltext table for papers containing your keywords.
More powerful than title/abstract search — finds methodological details
in the body of papers.

Args:
    query: Keywords to search for, space-separated (words shorter than three
        letters and common stop-words are ignored).
    max_results: Maximum number of results to return (default 10).

Returns:
    A single TextContent listing each matching paper's title (or filename)
    with a highlighted snippet around the first matched keyword, or a
    "no results" / "no usable keywords" message.

Interests should not depend on someone being able to articulate them from a
blank field. Their work already states them — project titles, captured ideas,
what their papers are about, what their sessions keep returning to. This mines
those sources for recurring domain terms and proposes the ones missing from
their declared lists.

Proposals go to the LIBRARY list by default: what recurs in someone's own
papers and projects is what they want a background on. What they want to hear
about in the NEWS is a different question, frequently outside their work
entirely, and cannot be mined from it — that one has to be asked. See
`start_interest_interview()`.

Args:
    apply: False (default) proposes only, so the researcher decides. True adds
        the suggestions to `library_interests` directly — use when they have
        said to go ahead, or for a silent top-up during onboarding.
    top: How many suggestions to consider. Default 12.

Returns:
    Suggested terms with what evidence supports each, and what was already
    declared so nothing is proposed twice.

Call this when the researcher wants help defining their research interests or
news topics — typically arriving from the "Set up my interests" button on the
Metis Systems surface, which opens a conversation asking for exactly this.

Returns the full interview to follow. It lives here rather than in the button's
deep link because a `claude://` URL carrying the whole script came to 3,222
characters, over the ~2,048-character limit Windows protocol handlers accept —
it would have been truncated or dropped silently. Keeping the script here also
means it is maintained in one place instead of duplicated in a template.

NEWS AND LIBRARY INTERESTS ARE SEPARATE THINGS, and either may sit entirely
outside the person's work:

  - `news_interests`    → what they want to know is HAPPENING. Drives the
    story-thread vocabulary (Metis groups news into persistent running
    stories, and a declared interest becomes a subject it can recognise), the
    News surface category tabs, and what the daily and weekly briefings
    monitor. Legitimately includes pure curiosity — following AI policy or a
    conflict closely without ever publishing on it.
  - `library_interests` → what they want a real SCIENTIFIC BACKGROUND on.
    Drives literature relevance and the knowledge layer. Slower and deeper,
    and also allowed beyond current projects: a method they intend to learn
    belongs here before it appears in any of their work.

They overlap, but the differences are the useful part. Do not collapse them.

Neither list requires the person to have any projects or articles. Someone can
join Metis with nothing set up and still have clear interests — ask, do not
infer. Call `start_interest_interview()` for the full interview.

Prefer specific, searchable terms: "sleeping sickness" and "DHIS2" work,
"global health" is too broad to filter anything.

Args:
    news_interests: What the News surface and briefings should track.
        Omit to leave unchanged.
    library_interests: What the literature collection and knowledge layer
        should build on. Omit to leave unchanged.
    role: Optional one-line role description.
    display_name: The name to address them by. Set it when someone new tells
        you their name — without it they are greeted as "Researcher".
    mode: "replace" (default) overwrites; "add" merges into what is there.
    interests: DEPRECATED alias for `library_interests` — the old combined
        field. Accepted so existing callers keep working.
    news_topics: DEPRECATED alias for `news_interests`.

Returns:
    What was saved per list, and what each one now affects.

Call this at the start of any personalised run to understand the user's
topics, news signals, and communication preferences.

Returns JSON with:
- display_name: user's display name (set via /metis_config)
- role: professional role (e.g. "Senior researcher")
- news_interests: what they want to know is HAPPENING — drives the News
  surface and the briefings. May include things outside their work.
- library_interests: what they want a scientific BACKGROUND on — drives
  literature collection and the knowledge layer. Also may sit outside their
  current projects.
- has_declared_interests: False if neither list is set. A new user may have
  no projects, articles or interests yet; that is a normal starting point,
  and `start_interest_interview()` is how to fill it in.
- interests: DEPRECATED — the union of both lists, kept for older callers
- news_topics: DEPRECATED — mirrors news_interests
- active_model: current default model slug (haiku / sonnet / opus)
- style: dict of communication preferences:
    - response_length: "concise" | "moderate" | "detailed"
    - feedback_style: "gentle" | "direct" | "challenging"
    - challenge_level: "supportive" | "balanced" | "rigorous"
    - warmth: "warm" | "neutral" | "formal" (default: "warm")
    - detail_level: "brief" | "balanced" | "thorough" (default: "balanced")
    - routing_verbosity: "silent" | "natural" | "detailed" (default: "natural")

Usage pattern:
  profile = json.loads((await get_user_profile())[0].text)
  interests = profile['interests']
  style = profile['style']  # → {"response_length": "concise", "warmth": "warm", ...}

Uses NCBI E-utilities (free, no API key required). Results are inserted
into news_briefs with source_type='article'. Safe to call daily from the
morning scan scheduler job.

Args:
    query: PubMed search query. Defaults to the query in user-preferences.json
           (pubmed_query field), then to your configured research field from
           user-config.yaml, then to a generic global-health fallback.
    reldate: Look back this many days (default: 1 = yesterday + today).
    max_results: Maximum papers to retrieve (default: 15).

OpenAlex covers 474M papers including preprints. Free API, no key required.
Results are inserted into news_briefs with source_type='article'.

Args:
    query: Free-text search query. Defaults to the query in user-preferences.json
           (openalex_query field), then to your configured research topics from
           user-config.yaml, then to a generic global-health fallback.
    days_back: How many days back to search (default: 1).
    max_results: Maximum papers to retrieve (default: 10).

Complements PubMed (scan_pubmed_alerts) and OpenAlex (scan_openalex) with
Semantic Scholar's 200M+ paper graph and citation counts. Free public Graph
API, no key required. Matched papers are inserted into news_briefs with
source_type='article' (domain 'Semantic Scholar') so the Librarian and the
dashboard surface them like any other discovered paper.

Args:
    query: Free-text topic to search (e.g. "HAT elimination surveillance").
           Defaults to your configured PubMed/topic query if empty.
    max_results: Maximum papers to retrieve (default 10, max 100).

Call this after saving a meeting transcript (via Meetings tab or
transcribe_recording()). It extracts key topics from the transcript,
matches them against tasks, library papers, and active projects, and
returns a structured cross-reference brief.

The result is also written to the meeting's notes field in the database
so it appears in the Meetings tab.

Args:
    meeting_id: The meeting_id from the meetings table.

Returns:
    Formatted cross-reference brief listing matched tasks, papers, and projects.

Creates a course record in the course_builds table, adds a placeholder
row to learning_courses (status='building'), and returns the intake
questionnaire for the user to complete.

Args:
    topic: The subject or title of the course (e.g. "Multilevel models for epidemiologists")
    target_audience: Who this course is for (e.g. "MPH students with basic R knowledge")
    duration_hours: Estimated total course length in hours (0 = TBD)
    notes: Any initial notes or constraints the user has mentioned

Returns:
    Course ID, a brief confirmation, and the intake questionnaire.

Call this once the user has reviewed and approved the module outline.
Pass outline_json as a JSON array of module objects:
  [{"module": 1, "title": "...", "bloom_level": "...", "hours": 2}, ...]

Args:
    slug: The course slug returned by start_course_build()
    outline_json: JSON array of module definitions
    approved: Must be True to advance the build to Step 3 (Harvest)

Returns:
    Confirmation and next-step instructions.

Args:
    slug: Course slug. Leave empty to list all active builds.

Returns:
    Status summary including current step, modules, and next action.

Call this after the Content Harvester has collected materials. Pass a
JSON array of source objects. Each source is written as a YAML file in
`knowledge/courses/{slug}/sources/` and the build advances to Step 4.

Args:
    slug: The course slug.
    sources: JSON array of source dicts — each must have at least a
             ``title`` and one of ``url``, ``file_path``, or ``doi``.
             Example: '[{"title": "OpenIntro Stats", "url": "https://openintro.org/book/os/", "type": "textbook"}]'

Call this after the Learning Architect has produced the curriculum.
Pass a JSON object with ``modules`` (array) and ``lessons`` (array).
Writes to `knowledge/courses/{slug}/course.json` and advances to Step 5.

Args:
    slug: The course slug.
    curriculum_json: JSON object with ``modules`` and ``lessons`` arrays.

Writes a single lesson's markdown into the course's lessons folder; call it
once per lesson during drafting. The content is validated and rejected
unless it contains all required sections, in order:
## Learning objectives, ## Prerequisites, ## Content (with ### Section N:
subsections), ## Summary, ## Exercises, ## Further reading. The filename is
derived from the lesson number and its title in lessons.json.

Args:
    slug: The course slug; selects the
        knowledge/courses/<slug>/lessons/ folder to write into.
    lesson_id: The lesson id, which must match an id in lessons.json
        (e.g. "lesson-01"); used to look up the title and build the filename.
    content: The full markdown body of the lesson, including all required
        sections listed above.

Returns:
    A confirmation with the written file path, or a rejection message
    listing the required sections that are missing.

Checks:
  - All lessons in lessons.json have a corresponding file on disk
  - Each lesson file contains the required section headers
  - No lessons are empty (< 200 chars)
  - lessons.json is valid JSON with modules and lessons arrays

Returns a pass/fail report. Fix any failures before calling publish_course().

Args:
    slug: The course slug to review.

Marks the learning_courses row as 'active', sets progress to 0,
and writes a completion note. Call after all 7 steps are done.

Args:
    slug: The course slug to publish.

Returns:
    Confirmation with the course path and Learning tab link.

Walks knowledge/library/ (all subdirectories), collects PDFs, and indexes
them with PaperQA2 + Claude Haiku. The index is persisted so that
ask_library() can query it without re-reading every PDF.

Run once after adding new papers, or with force_reindex=True to rebuild.

Args:
    force_reindex: Rebuild from scratch even if an index already exists.
    topic_filter: If given, only index PDFs whose parent folder name
                  contains this string (e.g. "NTD", "Epidemiology", "Methods").
    scope: Which library to index. "default" = knowledge/library/ (all PDFs).
           "ph_library" = knowledge/library/ph-background/ (public health background).

Searches the pre-built index (see index_library_pdfs()) and returns
a synthesised answer with citations from the source papers.

Args:
    question: Natural language question to answer from the library.
    top_k: Number of source passages to retrieve before synthesis (default 5).
    scope: Which index to query. "default" = full library. "ph_library" = PH background only.
           Build the index first with index_library_pdfs(scope=<scope>).

Elicit-style structured extraction: runs a fixed question set over the indexed
library (PaperQA2) and assembles a markdown table — one row per field, each
answer carrying its citations. Useful for systematic-review scaffolding.

Args:
    topic: The subject to extract on (e.g. "HAT passive screening sensitivity").
    fields: Optional comma-separated fields to extract. Default set covers
            population, design, sample size, outcome, finding, limitations.
    scope: Which index to query ("default" or "ph_library"). Build it first
           with index_library_pdfs().

Uses faster-whisper locally — entirely offline, no API calls, no data leaves
your machine. Supports MP3, WAV, M4A, OGG, FLAC, and most other audio formats.

Args:
    audio_path: Path to an audio file to transcribe. Leave empty when using
                record_seconds for live mic capture.
    route_to:   What to do with the transcript after transcription:
                - "raw"     → return the transcript text only (default)
                - "idea"    → capture as an idea with cross-pollination
                - "journal" → add as a journal entry (mood + energy auto-extracted)
                - "note"    → append to today's voice-notes markdown file
    record_seconds: Seconds to record from the microphone. Requires sounddevice
                    and numpy. Only used when audio_path is empty.
    model_size: faster-whisper model size. Options: "tiny", "base", "small",
                "medium", "large-v3". Defaults to METIS_WHISPER_MODEL env var,
                or "base". Larger models are more accurate but slower to load.

Returns:
    The transcript text and (if routed) confirmation of where it was saved.

Examples:
    transcribe_voice(audio_path="/tmp/idea.m4a", route_to="idea")
    transcribe_voice(audio_path="/tmp/reflection.mp3", route_to="journal")
    transcribe_voice(audio_path="", record_seconds=30, route_to="idea")
    transcribe_voice(audio_path="/tmp/note.wav", route_to="note")

A disabled layer is skipped when a question searches "all" layers, so its
documents stop influencing answers — useful when a background is off-topic for
the work at hand. It is not deleted, and naming it explicitly in a search still
reaches it.

Args:
    database: The layer's slug (e.g. 'ph-background', 'hat-specialist').
    enabled: True to include it in searches, False to exclude it.

Deletes the layer's registry row, its indexed passages and its vectors. Your
PDFs on disk are NOT touched — only the search index built from them, so the
layer can always be rebuilt by pointing at the same folders again.

Args:
    database: The layer's slug (e.g. 'my-custom-layer').
    confirm: Must be True to proceed. Without it, this reports what WOULD be
        deleted and changes nothing.

Uses local nomic-embed-text-v1.5-Q (fastembed, ONNX) — no API key required.
Each database is a named layer: 'ph-background', 'hat-specialist', 'epi-methods',
or any custom database slug created via create_knowledge_database().

Args:
    database:      Slug of the knowledge database to build (default: 'ph-background').
    force_rebuild: Re-index even files already indexed in this database.

Meaning-based (vector) search over your knowledge-base PDF chunks — the RAG
retrieval tool behind grounded, cited answers. For exact keyword matches use
search_fulltext; for reference metadata use search_library; for your own
notes/memory (not documents) use semantic_search.

Searches indexed PDF chunks using 768-dim nomic-embed vector similarity.
You can search a single layer or combine layers (e.g. PH background + HAT specialist).

Args:
    query: Natural language question or keyword phrase to embed and match.
    databases: List of database slugs to search; pass None (the default) to
        search all indexed databases. Examples: ['ph-background'],
        ['hat-specialist', 'epi-methods'].
    top_k: Number of results to return (default 8).

Returns:
    A single TextContent listing the top-ranked PDF chunks (title, similarity
    score, layer, domain, page, source file, and an excerpt), or a message if
    nothing is indexed yet or no chunks match the requested databases.

Use this to check how complete your RAG (retrieval) corpus is before
relying on grounded answers: it shows, per knowledge database, the document
and chunk counts, when it was last built, a domain breakdown, and which PDFs
are still un-indexed. A quick "is my library ready to answer questions?"
health check. Pairs with index_pdf_library (to index) and
search_pdf_knowledge (to query).

Args:
    database: Slug of a single knowledge database to report on (e.g.
        "ph-background"). Empty string (the default) reports on every
        database.

Returns:
    A formatted status report covering each database's counts, last-built
    time, domain breakdown, and any un-indexed PDFs.

After creating, add PDFs to knowledge/library/<your-folder>/ and call
build_pdf_knowledge_db(database='<slug>') to index them.

Args:
    slug:        URL-safe identifier (e.g. 'dhis2-specialist', 'malaria-research').
    name:        Human-readable name (e.g. 'DHIS2 Specialist Knowledge').
    description: What this database covers.
    layer:       Layer number (4+ for custom; built-ins use 1–3).
    folders:     List of library subfolder paths to include
                 (e.g. ['open-access-books/Health Informatics & DHIS2']).

The sweep behind "I dropped a PDF into the folder". Pack installs and Zotero
imports schedule their own indexing, but a file copied in by hand — which is
what _TO-OBTAIN.md asks for — has no code path to notice it. This closes that
gap, and is what the nightly job should call.

Runs detached and single-flight: a layer already indexing is left alone rather
than started twice.

Args:
    confirm: False to report what is pending without starting anything.

Without confirm=True this only reports what WOULD be downloaded — a pack fetches
files from the internet, so the researcher should see the list and the hosts
before it runs.

Indexing is NOT run here; it happens on the next nightly background index, or
immediately via the Rebuild button on the Library surface.

Args:
    slug: The pack's slug, from list_background_packs().
    confirm: True to actually download and register.

Produces the `pack.json` another machine (or another person) can install from.
It records the layer's identity and the documents it contains BY NAME; it does
not copy the PDFs, and it cannot invent a download URL for a file that came off
your own disk — those entries are written with an empty url and must be filled
in before the pack can be installed elsewhere. Saying so is the point: a
manifest that silently omitted them would look complete and install empty.

Args:
    database: The layer's slug (e.g. 'ntd').
    out_dir: Where to write it. Defaults to backgrounds/<slug>/ in the repo.

Deduplicates by normalised fingerprint, so a decision restated across many
sessions becomes ONE row with a times_seen count rather than many rows. Rows
already resolved are left alone — re-running this never reopens a closed call.

Args:
    limit: How many recent session summaries to scan.

Ordered by how often a decision has been restated, because the one you keep
circling is the one costing you most.

Args:
    limit: How many to show.
    state: open | agreed | rejected | deferred | dropped | all

An agreed decision that reads as a standing rule is also copied into
user_decisions, so it starts being honoured rather than merely recorded.

Args:
    od_id: From review_open_decisions().
    state: agreed | rejected | deferred | dropped
    why:   One line on the reasoning — this is the part you cannot reconstruct later.

PowerPoint: returns the slides — titles, body text and speaker notes — because
a deck is the researcher's own authored content.

Excel: returns STRUCTURE ONLY — sheet names, row/column counts, column headers
and inferred types. Never cell values. If you need the data analysed, use the
safe-analysis procedure: Metis writes a script, the researcher runs it locally,
and only aggregates come back.

Args:
    path: Path to a .pptx or .xlsx file, absolute or relative to the Metis folder.

Stores the deck's content so it can be recalled and cross-referenced later —
which is what closes the round trip: a deck edited in PowerPoint becomes
something Metis knows about again.

Excel files are NOT ingested. Their structure can be read (see
read_office_document) but their contents must not enter memory — they are data,
not authored content, and may hold patient records.

Args:
    path: Path to a .pptx file.
    project_id: Optional project to attach it to.

Call this at the end of any substantive session so future sessions can
recall what was discussed, decided, or built — the core of Metis's
cross-session continuity. The saved summary is searchable later via
search_session_memory and surfaces when you resume related work.

Args:
    summary: A 2–5 sentence, plain-English summary of what happened this
        session. Required.
    key_topics: Optional list of short topic tags for retrieval
        (e.g. ["phase-10", "APScheduler"]).
    decisions: Optional list of key decisions made
        (e.g. ["switched to AGPL-3.0"]).
    session_id: Optional identifier used to group related summaries; if
        omitted, the summary is stored on its own.
    client: Which Claude client this session ran in — 'code' (Claude Code),
        'chat' (Claude Desktop), or 'dashboard'. Lets the dashboard show a
        Code-vs-Desktop breakdown. Optional; defaults to unset.

Returns:
    A dict with the saved record's id and a confirmation status.

Use this to recall what was discussed in previous sessions — e.g.
"what did we decide about the installer?" or "when did we build APScheduler?".

Args:
    query: Keyword or phrase to search for.
    limit: Maximum number of results to return (default 10).

This is the mandatory recording checkpoint — call it at the end of every
agent-routed session, before delivering the final result to the user.
Unlike save_session_summary (which accepts any summary), this tool enforces
that at least one concrete decision is captured, and writes each decision
separately to episodic_memory for future retrieval.

Args:
    decisions: 1-5 plain-English decisions made this session. Be specific:
        "Chose logistic regression over mixed model due to data sparsity in
        Zone de Santé X" not "made a modelling decision".
    summary: 1-3 sentence summary of the session context (optional but useful).
    key_topics: Topic tags e.g. ["DHIS2", "domain surveillance", "tracker design"].
    session_id: Optional session identifier.

Returns the rolling history of saved session summaries so you can pick up
where a previous conversation left off or review recent decisions and
topics. Each entry carries its summary, key topics, and decisions.
Complements search_session_memory (keyword search) and
save_session_summary (which writes these rows).

Args:
    limit: Maximum number of summaries to return, most recent first
        (default 20).

Returns:
    A list of session-summary dicts (id, session_id, summary, key_topics,
    decisions, created_at); a single-item list with an "error" key on failure.

Reads the n most recent agent_runs, identifies runs with output files and
substantive task summaries, deduplicates against existing memory_entries,
and writes new entries to the DB + markdown files.

Args:
    n_runs: Number of recent agent runs to review (default 20).
    min_quality: 'high' = only runs with output files; 'all' = include run-only entries.

Tiered retention keeps recall() fast by compressing old raw entries into
distilled summaries while preserving the knowledge:

1. **Episodic decisions** older than decision_age_days (default 90) are
   grouped by month and project, summarised into a single consolidated
   entry per group, and the originals are marked archived=1 (excluded
   from default recall but still queryable).

2. **Session summaries** older than session_age_days (default 180) are
   marked archived=1 so they don't clutter keyword search.

3. **Reflexions** are consolidated monthly — patterns are distilled
   into a single "what we learned in month X" entry.

Run with dry_run=True first to see what would happen without changes.

Args:
    decision_age_days: Archive episodic decisions older than this (default 90).
    session_age_days: Archive session summaries older than this (default 180).
    dry_run: If True, report what would be done without making changes.

Returns:
    A report of consolidation actions taken (or planned if dry_run).

Searches memory_entries by topic keyword and optional tag list, ranks by
relevance, and formats the top N entries for injection into the current
agent's working context.

Args:
    topic: The topic or task description to search for.
    tags: Optional comma-separated topic tags to include (e.g. 'methods,phd').
    top_n: Maximum number of entries to return (default 5).

Returns: entry counts by type, topic coverage map, coverage gaps vs active
projects, duplicate candidates, and entries without output file provenance.

The unified front door to everything Metis remembers — personal notes,
agent findings, project decisions, PDF knowledge, ideas, and session
history. Returns ranked results with source attribution.

Searches up to 6 layers and merges results using Reciprocal Rank Fusion:
  1. Memory palace (memory_entries — keyword)
  2. Episodic memory (events — vector + keyword)
  3. Semantic memory (concepts — vector + keyword)
  4. Procedural memory (workflows — vector + keyword)
  5. Knowledge databases (PDF chunks — vector)
  6. Ideas (ideas table — keyword)

Use scope/agent_id/project_id to narrow results:
  recall("HAT diagnostics")                          → everything
  recall("search patterns", agent_id="librarian")    → librarian's wisdom
  recall("elimination", project_id="article-1")      → Article 1 decisions
  recall("mixed models", scope="global")             → cross-cutting knowledge

Args:
    query: Natural language search query.
    scope: Filter by scope: 'global', 'agent', 'project', 'session'.
        Empty string (default) searches all scopes.
    agent_id: Filter to memories from a specific agent (e.g. 'librarian').
    project_id: Filter to memories linked to a specific project.
    layers: Comma-separated layers to search. Options: 'memory', 'episodic',
        'semantic', 'procedural', 'knowledge', 'ideas', or 'all' (default).
    top_k: Number of results to return (default 10).
    recency_half_life: Days until recency boost halves (default 90). Set to 0
        to disable recency weighting. A 90-day-old entry gets ~37% of a
        fresh entry's time bonus; 180-day ~14%.

Returns:
    Ranked results from across all matching layers, each with its source
    layer, relevance score, timestamp, and content preview.

The unified write interface to Metis's memory system. Classifies the content
into the appropriate memory layer and stores it with scope tags for later
retrieval via recall().

Memory types:
  - 'episodic': A time-stamped event (something that happened)
  - 'semantic': A distilled concept or definition (timeless knowledge)
  - 'procedural': A workflow or how-to pattern
  - 'note': A human-curated memory palace entry (saved to DB + markdown)
  - 'auto': Let Metis classify based on content (default)

Scope tags determine visibility:
  - scope='global': Visible to all agents and projects
  - scope='agent': Visible only when agent_id matches
  - scope='project': Visible only when project_id matches
  - scope='session': Visible only in this session

Args:
    content: The text to remember. Can be a finding, concept, workflow,
        decision, or any text worth preserving.
    memory_type: How to classify this memory. One of 'episodic', 'semantic',
        'procedural', 'note', or 'auto' (default — Metis decides).
    agent_id: Tag this memory as belonging to a specific agent (e.g. 'librarian').
    project_id: Tag this memory as belonging to a specific project.
    scope: Visibility scope: 'global' (default), 'agent', 'project', or 'session'.
    title: Optional short title. Auto-generated from content if empty.
    tags: Comma-separated topic tags (e.g. 'hat,diagnostics,rdts').
    session_id: Current pipeline session ID (optional).

Returns:
    Confirmation of what was stored, in which layer, with what scope.

Links two entries across any memory layers — episodic, semantic, procedural,
memory (palace), or knowledge. This builds a lightweight knowledge graph on
top of the existing memory layers, enabling graph traversal during recall.

Example relations: 'informed_by', 'contradicts', 'elaborates', 'derived_from',
'supersedes', 'related_to', 'supports', 'led_to'.

Args:
    source_layer: Layer of the source entry ('episodic', 'semantic',
        'procedural', 'memory', 'knowledge').
    source_id: Row ID of the source entry in its layer.
    target_layer: Layer of the target entry.
    target_id: Row ID of the target entry in its layer.
    relation: Relationship type (e.g. 'informed_by', 'contradicts').
    note: Optional free-text note explaining the relationship.

Returns:
    Confirmation of the created link, or an error if it already exists.

Traverses the lightweight knowledge graph to find connected entries.
Use this after recall() to explore how a result connects to other
knowledge — e.g. which papers informed a decision, or which concepts
relate to a workflow.

Args:
    layer: The layer of the entry to look up ('episodic', 'semantic', etc.).
    entry_id: Row ID of the entry.
    direction: 'outgoing' (this entry → others), 'incoming' (others → this),
        or 'both' (default).

Returns:
    List of related entries with their relationship type and content preview.

Gathers information about the research structure, articles, milestones,
and methods to help with research planning and writing.

Args:
    section: What to retrieve -- "overview", "articles", "milestones", "methods".
    max_chars: Maximum characters to return for file-based sections (default 8000).
               Pass 0 for no limit.

Use this whenever you reach a conclusion, update a previous belief, or
encounter evidence that changes your view. The timeline preserves the full
chain of reasoning across sessions.

Args:
    entity: What this claim is about. Use a consistent name across sessions
        (e.g. "RDT sensitivity in low-burden areas", "Disease X elimination study",
        "DHIS2 tracker performance").
    claim: Your current belief or finding in 1-3 sentences.
    evidence: What supports this claim — paper citation, data result, meeting
        discussion. Brief reference is enough.
    confidence: "low", "medium", or "high".
    source_type: "session", "paper", "meeting", "data_analysis", "literature_review".
    source_ref: Specific reference — DOI, file path, meeting date.
    supersedes_id: If this replaces a previous claim, pass that claim's id.
        Set to 0 if this is a new claim with no predecessor.

Returns all claims for an entity ordered by date, showing how thinking
evolved. Superseded claims (older beliefs you updated) are hidden by default
but can be shown to trace the full reasoning chain.

Args:
    entity: Filter by entity name (partial match). Leave empty to see all
        recent entries across all entities.
    since_date: ISO date string (YYYY-MM-DD). Only show entries on or after
        this date. Leave empty for all time.
    show_superseded: If True, include claims that have been replaced by newer
        ones. Default False — shows only the current belief for each topic.

Use this to get an overview of what topics have tracked beliefs, before
drilling into a specific entity with query_research_timeline().

Checks: git commits, modified files, todo completions, new documents.
Updates the project's scan_summary and last_scanned fields.
Also refreshes CLAUDE.md in the project folder.

Args:
    folder_path: Absolute path to the project folder.
    scan_type: One of:
        "names"   — file/folder names only (fast, no content read)
        "content" — reads README, CLAUDE.md, PLANNING.md (more accurate)
        "none"    — skip scan, return empty (user will describe manually)

Args:
    project_id: The Metis project ID to register.
    write_config: If True, write directly to Claude Desktop config.
                  If False, return instructions for manual setup.

Lists the user-defined categories (e.g. "Article", "Grant", "Teaching")
that have been assigned to projects, so you can group projects or offer
an existing category before set_project_category invents a new one.

Takes no arguments.

Returns:
    A text line listing the distinct non-empty categories in alphabetical
    order, or a message that none are defined yet.

Sets the grouping label on a project so it sorts with related work on the
dashboard. Call get_project_categories first to reuse an existing label
rather than creating a near-duplicate.

Args:
    project_id: The project_id of the project to update.
    category: The category label to assign (e.g. "Article", "Grant");
        surrounding whitespace is trimmed.

Returns:
    A confirmation message naming the category and project that were set.

This is the unified project creation tool used by all installers and the dashboard.

Args:
    title:        Project name.
    folder_path:  Absolute path to the project folder on disk.
    category:     User-defined category (e.g. 'Article', 'Grant', 'Teaching').
    description:  What this project is about. If empty and scan_type != 'none',
                  auto-detected from folder.
    scan_type:    'names' | 'content' | 'none' — how to infer description from folder.
    link_claude_desktop_auto: Write project to Claude Desktop config automatically.

Returns the JSON response as formatted text. Use for live metadata
validation, data element lookup, indicator queries, and data quality checks.

Args:
    endpoint: API path relative to /api/, e.g. "dataElements" or "organisationUnits.json".
              If it does not start with "/api/", that prefix is added automatically.
    params:   Query parameters as a dict, e.g. {"fields": "id,name", "paging": "false"}.
    method:   HTTP method — "GET" (default), "POST", or "PUT".
    body:     Request body for POST/PUT (serialised to JSON).

Examples:
    dhis2_query("dataElements", {"fields": "id,name,valueType", "paging": "false"})
    dhis2_query("system/info")
    dhis2_query("organisationUnits", {"filter": "level:eq:2", "fields": "id,name,level"})

Convenience wrapper around dhis2_query() for common metadata lookups.

Args:
    resource: Metadata resource type — e.g. "dataElements", "indicators",
              "organisationUnits", "programs", "dataSets", "trackedEntityTypes".
    filters:  Filter expressions in DHIS2 format, e.g. ["name:ilike:malaria", "valueType:eq:NUMBER"].
    fields:   Comma-separated field list (default: "id,name,shortName").
    paging:   Set True to get only the first page (faster for large resources).

Examples:
    dhis2_metadata("dataElements", ["name:ilike:HAT", "valueType:eq:INTEGER"])
    dhis2_metadata("programs", fields="id,name,programType,trackedEntityType[id,name]")
    dhis2_metadata("organisationUnits", ["level:eq:3"], fields="id,name,level,parent[id,name]")

Call at natural trigger moments (user starts a project, writes R code, builds a
library, asks a knowledge question, handles a dataset…). Pass `context` as
comma-separated trigger tags. Returns at most one tip — and ONLY for a feature the
user does NOT already use (earned discovery), respecting the off/snooze/power-user
settings and a frequency cap (≤1 tip / 20 min, ≤3 / day). Records it so it never
repeats. Returns '' when nothing should be shown. Weave the tip in naturally.

Args:
    context: comma-separated trigger tags describing what the user is doing.

Use ONCE for a brand-new user (or when they ask 'what can you do?'). After this,
rely on `next_discovery_tip` for the long tail. No-op (returns '') if the intro was
already given or tips are off.

Args:
    enabled:     True/False to turn tips on/off entirely.
    power_user:  True = expert mode (tips stay quiet); False = guided mode.
    snooze_days: >0 to snooze ALL tips for N days ("remind me later").

Tells you whether the just-in-time feature tips are on or off, the current
mode, any active snooze, and how many tips have been shown versus how many
of those features the user has since started using (adoption). Use it to
answer "are tips on?" or to sanity-check before changing them with
set_discovery_tips. Pairs with discovery_intro and next_discovery_tip.

Takes no arguments.

Returns:
    A one-line text summary: on/off, mode, snooze note, shown count, and
    how many shown features are now adopted.

Use this whenever the user writes or shares code worth reusing, so it can be
found and rebuilt later. Capture as much reproducibility context as you can.

Args:
    title: Short descriptive name (e.g. "INLA BYM2 spatial model setup").
    code: The actual code.
    language: r | python | sql | stata | shell | … (lowercase).
    project_id: The project this belongs to (links it for reuse). Optional.
    kind: script | snippet | function | template.
    purpose: One line on what it does / when to use it.
    tags: Comma-separated tags (e.g. "spatial,INLA,mapping").
    file_path: Where the script lives on disk, if any.
    packages: Dependencies / environment (e.g. "INLA 24.9, sf, dplyr").
    params: Seeds, thresholds, hyperparameters used (for exact reproduction).

Captures each variable's name, type, label, unique values / factor levels,
and units so future analysis scripts reuse the exact same names and
treatments. Re-registering the same dataset is idempotent: it replaces the
previous dictionary for that dataset (matched on dataset_name + project_id).

Args:
    dataset_name: Name of the dataset, e.g. "hat_cases_2015_2023".
    variables: List of variable entries. Each entry may be a plain string
        (the variable name) or an object with any of: name (required),
        type, label, unique_values, units, notes. Entries without a name
        are skipped.
    project_id: Project this dataset belongs to (default empty string);
        also part of the key used when replacing an existing dictionary.
    dataset_path: Where the dataset lives on disk (default empty string).

Returns:
    A confirmation message with the count of variables recorded for the
    dataset, or an error if none were provided.

Build a traceable chain raw → cleaned → analysis dataset, so any result can
be reproduced. Call once per step (recode, filter, join, derive, …).

Args:
    dataset_name: The dataset being transformed.
    description: What this step does (e.g. "drop records with missing age").
    project_id: Project this belongs to. Optional.
    step_type: recode | filter | join | derive | clean | other.
    code: The code for this step, if any.
    input_dataset: Dataset(s) this step consumes.
    output_dataset: Dataset this step produces.

Finds reusable scripts/functions, dataset variables and cleaning steps that
match a query — e.g. "Poisson model offset" or "catastrophic expenditure".

Args:
    query: What you're looking for.
    project_id: Restrict to one project. Optional.
    language: Restrict to a language (r, python, …). Optional.

Pulls the most relevant prior code, the project's dataset variables/paths,
and the cleaning steps — so you can write a new script in the user's own
conventions (same names, paths, packages). Call this, then write the script.

Args:
    goal: What the new script should do.
    project_id: The project to scaffold for (prioritised, then cross-project).
    language: Target language (r, python, …).

Returns packages, file reads/writes, variable names, source dependencies,
and transform patterns — all from regex parsing (no execution, no AST,
no data access).  The script's code is read but never sent to an LLM;
only the extracted metadata is returned.

Args:
    path: Absolute path to the script file (.R, .Rmd, .qmd, or .py).

For each script found (.R, .Rmd, .qmd, .py):
- Parses packages, file reads/writes, variables, and transforms
- Auto-registers each as a code_artifact in the Code Repository
- Aggregates variables across all scripts
- Maps detected file paths to dataset names

Returns a summary: N scripts, M packages, K datasets, P variables.

Args:
    folder_path: Absolute path to the folder to scan.
    project_id:  Project ID to associate the scripts with (optional).

The generated script runs on the user's machine and reports ONLY
aggregates: variable names, types, unique counts, ranges, missingness,
and value frequencies (capped at 30 levels).  No row-level data leaves
the machine.  Output is structured JSON matching the register_data_dictionary
schema, ready for ingest_profiling_output().

If data_dictionary entries already exist for this dataset, the script
includes checks for those specific variables.

Args:
    dataset_name: Name of the dataset (used in filenames and DB).
    project_id:   Project ID for looking up existing variable info.
    language:     "r" or "python" (default "r").
    dataset_path: Path to the data file on disk. If empty, a placeholder is used.

The user runs the profiling script locally (generated by
generate_profiling_script), which produces a JSON file.  This tool
reads that JSON and calls register_data_dictionary logic to insert/replace
the variable definitions.

Args:
    json_path:    Absolute path to the profiling JSON output file.
    dataset_name: Override the dataset name (default: read from JSON).
    project_id:   Project to associate the dictionary with.

Most Metis tools are kept out of the active set to save context; this finds the
ones you need by keyword (e.g. "dhis2 tracker", "clean dataset", "backup",
"knowledge graph", "transcribe"), makes them callable, and returns their names.
After calling this, call the returned tools directly as normal.

Args:
    query: Keywords describing what you want to do.
    limit: Max tools to return/load (default 8).

Use this when you know you'll do several related operations — e.g. group
"data" for a full dataset-cleaning session, "specialist" for DHIS2 work.

The tool that closes the largest gap in Claude Desktop: Metis could read
presentations but never produce one, so any "make me a deck" request had no
way to land as a file.

Args:
    title: Deck title, used on the title slide and in the filename.
    slides: The deck as plain text. Each slide starts with a line beginning
        "# " (the slide title); every following line becomes a bullet.
        A line beginning "> " becomes speaker notes for that slide.
        Blank lines are ignored. Example:

            # Why passive surveillance underperforms
            Health-facility detection depends on care-seeking
            Median delay to diagnosis exceeds 12 months
            > Open with the Mitashi 2014 figure

            # What changes with RDTs at the periphery
            Sensitivity holds at lower prevalence
        A line beginning "## " starts a section-divider slide instead.
    subtitle: Optional subtitle for the title slide.
    filename: Optional filename; derived from the title when omitted.
    template: The researcher's slide template stored in Metis (name or part
        of it). Empty = their default deck template if one is stored (see
        list_templates) — never ask them for the file. "none" = plain deck.
    closing: Text for the template's closing slide under "Thank you!" —
        e.g. presenter name, e-mail, website. Empty = no closing slide.

Returns:
    The path written and a slide count.

Call this before making a poster, a presentation or a formal document. The
templates are kept inside Metis (a private, machine-local folder), so never
ask the researcher where a template is or to attach it: it is already here,
and create_poster / create_presentation / fill_document_template find it by
name or use the right one by default.

The poster template is stored in Metis (see list_templates) and used by
default — never ask the researcher for the file or where it is.

The template (.potx or .pptx) is looked up in system/config/local/templates/,
knowledge/templates/, or $METIS_TEMPLATES_DIR — or pass a path. Each layout's
section boxes are detected from its geometry, so any template with a title,
small section headers and large content boxes works.

Args:
    title: Poster title.
    sections: The poster body as plain text. Each section starts with a line
        beginning "# " (its heading); following lines are its text. Lines
        beginning "- " become points, "## " a block subtitle inside the box,
        anything else plain prose. Sections fill the boxes down the first
        column, then the next. Example:

            # Background
            - Why the question matters
            # Methods
            - Design, setting, data
            # Results
            - The main finding, with its number
    authors: Authors line, shown under the title.
    band: Optional text for the band below the title — affiliations, or a
        one-sentence key message.
    template: Template file name (or part of it) or a path. Empty = the first
        template found; the result says which one was used.
    layout: "auto" (smallest layout with enough boxes), or part of a layout
        name such as "one column" / "two column".
    filename: Optional output filename; derived from the title when omitted.

Returns:
    The path written, the template and layout used, and any section whose text
    is likely to overflow its box.

The researcher's templates (e.g. an ethics-committee cover letter) are kept in
Metis — see list_templates; never ask them for the file. Returns each
non-empty paragraph with its key (p12, h0 for the header). Then call
fill_document_template with the keys whose text should change.

Args:
    template: Template name or part of it. Empty = the first stored document template.

Call read_document_template first to get the paragraph keys. The template's
letterhead, styles and fixed wording are kept; only the listed paragraphs change.

Args:
    edits: JSON object mapping paragraph keys to their new text, e.g.
        {"p5": "Antwerp, 14 October 2026", "p19": "This study ...", "h0": ""}.
        An empty string empties that paragraph; null deletes it.
    template: Template name or part of it. Empty = the first stored document template.
    filename: Output name; derived from the template when omitted.

Returns:
    Where the filled document was saved, including a path that opens from Windows.

Call this at the end when the researcher asks to see, show or open what was
made. Each file opens in its own Windows application (PowerPoint, Word, the
browser). Only files under outputs/ can be opened.

Args:
    paths_to_open: One or more paths as returned by the tool that wrote them
        (e.g. "outputs/presentations/2026-10-14_x.pptx"), separated by
        newlines or commas. Windows paths are accepted too.

Use after drawing a figure as an artifact: pass the same self-contained HTML
(inline SVG, or a chart library loaded from a CDN). It is stored under the
project and appears in the Figures section of that project's panel on the
Work tab, where it renders live. Saving again with the same title replaces it.

Args:
    project_id: The project slug (as returned when the project was created).
    title: Short figure title, shown above it on the dashboard.
    html: The figure as HTML — a full page, or a fragment that will be wrapped.
    caption: One or two sentences on what the figure shows, including the
        denominator and data source.

Returns:
    Where it was saved and the dashboard link that shows it.

The general "save this properly" tool. Without it, anything Metis produces in
a chat client exists only in the transcript, which is not somewhere work can
be found again.

Args:
    title: Document title; heads the file and derives its name.
    content: Body text. Markdown headings and bullets are honoured in both
        formats.
    subdir: Folder under outputs/ (default "reviews").
    fmt: "md" (default) or "docx".

Returns:
    The path written.

For researchers working in a chat client, where there is no shell. Every
operation is idempotent, reversible and destroys nothing; there is no way to
pass an arbitrary command.

Args:
    operation: One of —
        status             what is running, what ran last, how big the corpus is
        restart-dashboard  restart the Metis dashboard
        scan-library       search every literature source now
        scan-news          refresh the news feeds now
        pickup-downloads   file papers downloaded through the browser
        rebuild-index      index PDFs added to a knowledge layer
        health-check       run the self-test

Returns:
    What happened, in plain language.

Call this before answering anything substantive, so the reply can be framed in
their own prior work instead of starting from nothing. This is what makes Metis
read as a second brain rather than a fresh chat: "this is your fifth session on
the dashboard since May" is only sayable if something counted.

Returns, where each exists:
  - overall span — recorded sessions, how long Metis has been running, and how
    much is stored (memories, concepts, procedures, papers)
  - this project — age, session count, current next step
  - prior sessions on the same topic, newest first
  - decisions already taken (so a settled question is not reopened)
  - procedures that already cover this (follow them rather than improvising)
  - library items and news threads on the topic
  - the most recent briefing and whether it was read

USE IT HONESTLY. Weave one or two genuine references into the answer; do not
recite the whole payload, and never invent continuity. Empty fields mean there
is nothing to say — say nothing rather than padding. A fabricated link makes
the memory layer feel decorative, which defeats the purpose.

Args:
    topic: The question or subject being worked on. Free text; salient terms
        are extracted for matching.
    project_id: Project slug to scope to (e.g. "metis-dashboard"), if known.
    limit: How many prior sessions to return. Default 4.

Returns:
    A compact readable digest, plus a ready-made one-line continuity summary.

Use this when the researcher corrects how something should be done, states a preference
between real alternatives, or confirms an approach — anything that should change
Metis' behaviour in FUTURE sessions rather than only this one.

This is what makes the persona grow. Metis' memory already accumulates facts
about the work; without this, none of it changed how she behaves. A correction
that is only remembered as having happened is not learned.

WHAT BELONGS HERE:
  - "Prefers the explicit mark-read button over auto-marking" (a real choice)
  - "Wants feed URLs verified live before being added" (a workflow rule)
  - "Do not use ANSI colour; emoji survive copy-paste" (an avoid)

WHAT DOES NOT:
  - Facts about the research or the code — that is memory (`remember`,
    `save_session_summary`).
  - One-off task details that will not recur.
  - Anything the researcher has not actually confirmed. Do not record an inference as a
    lesson; if unsure whether it is durable, ask, or leave it.

Args:
    lesson: The behavioural rule, stated imperatively and briefly — how Metis
        should act from now on. One or two sentences.
    category: One of voice, workflow, preference, avoid, domain.
    evidence: Why this is believed — quote or paraphrase what the researcher said, and
        when. An unexplained rule cannot be audited later.
    supersedes: Optional. Text from an earlier lesson this replaces, so the
        older one is marked rather than silently contradicted.

Returns:
    Confirmation, and how many lessons are now on record.

Read this at the start of a session, alongside `metis-persona.md`, so
corrections the researcher made previously actually apply. Also useful when he asks what
Metis has learned, or to audit whether a rule still holds.

Args:
    category: Optionally filter — voice, workflow, preference, avoid, domain.

Returns:
    The recorded lessons with their evidence and dates.

Verifies what can be verified WITHOUT a model: that the cited document is
indexed, that the cited page exists, that the claim's figures appear on that
page, and that any quoted string is actually present. It cannot hallucinate a
verdict because it contains no model.

A `supported` verdict means the deterministic checks passed. It does NOT mean
the passage supports the claim — that is entailment (Tier C, Critic's job).
For a source that is not in the corpus, use `verify_doi` to establish whether
it exists at all.

Args:
    claim: The sentence or statement being checked.
    source: Cited document title or filename, as written in the answer.
    page: Cited page number; 0 or omitted searches the whole document.
    quote: An exact quotation to locate, if the claim contains one.
    record: Append the verdict to the citation ledger (default True).

Returns:
    The verdict, what was tested, and which figures (if any) were missing.

Use this on a draft, a lesson, a report section, or anything about to be
written to disk. Extraction is deliberately conservative: it matches
citations that name a source and a page, plus bare DOIs. A false "unverified"
warning is more corrosive than a missed one, because it trains the reader to
ignore the report.

Args:
    text: The prose to scan.
    artifact_path: File this text belongs to, recorded in the ledger so
        "which of my outputs rest on unverified citations?" is a query.
    record: Append each verdict to the citation ledger (default True).

Returns:
    A per-citation verdict table and a summary count.

The honest denominator for every grounding claim. There are three states, not
two: verified against text, known-but-unquotable (metadata only, no indexed
full text), and genuinely absent. Without this number, "not found in your
corpus" reads as "not in your literature" when it may only mean "not indexed"
— and a verification rate reported against an unknown denominator is itself
the kind of overclaim this whole layer exists to prevent.

Returns:
    Per-layer document and chunk counts, plus the quotable fraction of the
    reference library.

Reads the ledger. This is the question the AI in Public Health course raised
and nothing could answer — 16 lessons shipped with "every citation is an
unverified lead" because verification results had nowhere to accumulate.

Args:
    limit: Maximum rows to list (default 25).
    artifact: Restrict to one artifact path; omit for everything.

Returns:
    Counts by verdict, and the worst offenders first.

Catches the classic fabricated citation: plausible authors, plausible journal,
no such paper. And the failure nothing in Metis could see before — citing a
paper that has since been retracted, which is worse than a wrong page number
because the claim looks perfectly sourced.

Metadata only: a DOI goes out, bibliographic fields come back. Nothing of the
researcher's leaves the machine, so this stays inside the local-first rule.

A `doi_resolved` verdict means the paper is real, NOT that it supports the
claim. That distinction is the whole point of tiering.

Args:
    doi: The DOI as cited (a full https://doi.org/… URL is fine).
    expect_author: Surname as cited, to check against the record.
    expect_year: Year as cited.
    expect_title: Title as cited.
    record: Append the verdict to the citation ledger (default True).

Returns:
    The verdict, the resolved record, and any retraction or correction flags.

Sweeps `pdf_index_state` and records a DOI for every indexed document it can
identify, so a layer can state its own provenance. Matches against the local
reference library first — that is free and covers most of the corpus — and
only calls Crossref when `resolve_online` is set.

This is the "verify at ingest, trust at read" half of the verification layer.
A corpus of known-real documents makes answer-time checking a containment
problem rather than a truth problem, and the cost is paid once per document
instead of once per claim.

Args:
    layer: Restrict to one layer slug (e.g. 'hat-specialist'); omit for all.
    resolve_online: Also query Crossref by title for documents the local
        library cannot identify. Metadata only; costs one request each.
    limit: Maximum documents to process in one pass (default 400).

Returns:
    Per-layer provenance counts and what changed in this pass.

Fact-checking a number is not the same as checking a citation. A figure can be
correctly cited and still be one of many the literature reports — this corpus
holds specificity estimates for HAT serology ranging from 72% to 99.7%, and a
single confident answer hides that entirely.

So this reports the SPREAD, never one value: every quantified statement found,
with its document and page, the qualifiers attached to it (sample size,
confidence interval, reference standard, population, setting, threshold), what
is MISSING from each, sentences nearby that complicate the figure, and how old
the evidence is. Deterministic — lexical retrieval and regex, no model, so it
can miss an oddly-phrased figure but cannot invent one.

Use it before answering any "what is the X of Y" question. Then use
`check_for_newer_evidence` to find out whether the corpus is current.

Args:
    question: The question as asked, e.g. "specificity of CATT in passive screening".
    layers: Comma-separated layer slugs to restrict to; omit to search all.
    show: How many individual findings to list (default 14).

Returns:
    Per-metric spread, the individual estimates with provenance and
    qualifiers, the qualifier gaps, and the evidence date range.

The second half of fact-checking a quantity. A spread computed from a library
whose newest paper on the topic is from 2014 is a 2014 answer — possibly
correct, possibly superseded, and the corpus cannot tell you which because it
does not know what it lacks.

Recency is computed LOCALLY first and reported whether or not you go online,
because knowing how stale you are is what decides whether the search is worth
making. Set `search_online=True` to query PubMed and OpenAlex for work newer
than the corpus — metadata only, nothing of the researcher's is transmitted.

Anything returned from a search is ATTRIBUTED, not quoted: a title and a DOI
that nobody has read. Never present it beside a corpus passage as if the two
were the same kind of evidence, and use `verify_doi` before citing one.

Args:
    question: The question, e.g. "specificity of CATT in passive screening".
    search_online: Query PubMed + OpenAlex for newer work. Ask first.
    limit: Maximum results per source (default 8).

Returns:
    The corpus date range for this topic and, if requested, newer candidate
    sources with their years and DOIs.

A focus is for a subject you want to stay CURRENT on, as opposed to a course
you finish or a project you deliver. It owns no content: it is a saved query
plus a place to write, so removing it later cannot remove anything.

`keyword_groups` is the important argument, and it is groups rather than a
list on purpose: OR within a group, AND across groups. "AI in health" is a
conjunction of two axes, and a flat list returns "Can AI ever be conscious?".

Args:
    title: Human name, e.g. "AI in Health & Epidemiology".
    keyword_groups: JSON list of lists, e.g.
        '[["artificial intelligence","machine learning","llm"],
          ["health","clinical","epidemi","surveillance"]]'
    subtitle: One line on what this focus is for.
    layers: Comma-separated knowledge-layer slugs to search for documents.
    overview: The standing narrative — what matters here and why.
    activate: Put it on the shelf immediately (max 3 active).

Returns:
    The created focus and its shelf state.

`active`    on the shelf and in the left navbar. At most 3 — a shelf with
            ten things on it is not a focus.
`following` defined and queryable, off the navbar. Attention moved on.
`archived`  read-only, no refresh, still openable.

Nothing is ever deleted, and nothing the focus SHOWED is affected by any of
these transitions: notes, ideas, papers and briefs live in their own tables
and keep their tags. Removing a lens cannot remove what you saw through it.

Args:
    slug: The focus slug.
    state: active | following | archived.

Returns:
    The new shelf layout, or an explanation of why the move was refused.

This is the piece that was going to be the AI course's introduction: the
orientation you want in front of you every time you open the subject, as
opposed to the feed, which changes daily.

Args:
    slug: The focus slug.
    overview: Markdown. Replaces the existing overview entirely.

Returns:
    Confirmation and the new length.

The text form of the surface — for Claude Desktop, which has no dashboard.

Lens tuning was otherwise blind: the only way to learn a lens returned nothing
was to build the focus, open it and find it empty. Worse, a conjunction that
returns zero says nothing about WHICH group is at fault.

So this reports each group's matches ALONE as well as combined. If group 1
matches 400 and group 2 matches 3 and together they match 0, group 2 is the
problem and no judgement is required to see it.

Args:
    keyword_groups: JSON list of lists — OR within a group, AND across groups.
        e.g. '[["dhis2","hmis"],["surveillance","district"]]'

Returns:
    Per-group and combined counts, sample matches, and a plain verdict.

Lower, not gone: declining demotes and folds, it never deletes. Nothing stops
being shown, or you could never audit what you were no longer being offered.

kind: nature-daily · nature-ai · nature-transl · nature-anthro · nature-cancer

Added 2026-09-10, when the feedback loop was finally wired. Until then every
verdict was filed and none of it changed a ranking, so this tool exists as
much to make the learning *auditable* as to report it: a preference model you
cannot inspect is one you cannot correct.

Pass no slug to see every decision and which specialist owns it. The `applied`
count is the signal that matters: a decision recorded and never applied is
indistinguishable from one that was never recorded.

Args:
    agent_slug: Restrict to one specialist; omit for all.
    limit: Maximum rows.

Returns:
    The decisions, grouped, with their categories and apply counts.

Call this the moment the researcher says they like something. It is the cheapest of the
three writes and the one that makes the other two honest: a recipe whose
origin is unrecorded cannot be checked against the thing it came from.

Args:
    title: What the piece is called.
    what_you_liked: In his own words where possible. The most valuable field
        in the row — "I like it" is not reusable, "the labels sit on the
        marks instead of in a legend" is.
    url: Where it lives.
    source: Publication, team or author.
    published: Date of the ORIGINAL, not of saving.
    kind: chart | map | diagram | flow | table | layout.
    screenshot_path: Local image, if the page cannot be refetched later.
    unverified: Anything in this row written from memory rather than a read.
        Say so here; a library that launders recollection into fact is worse
        than an empty one.

Returns:
    The stored exemplar and its slug — use that slug verbatim as
    `derived_from`, since slugs are truncated to 60 characters.

A recipe is what makes a figure work: what the mark represents, what maps to
which channel, and what data shape it demands. Not the colours.

Args:
    name: Descriptive, e.g. "unit-dot transition flow".
    kind: chart | map | diagram | flow | table | layout.
    one_liner: What it shows, and why that framing beats the obvious one.
    encoding: What maps to position, colour, size, order.
    mark_unit: What ONE mark is. Usually the decisive choice — "one dot =
        one person" and "one bar = a percentage" are different arguments
        about the same numbers.
    data_contract: JSON object of role -> meaning, e.g.
        '{"unit_id":"one row per person","start_class":"origin group",
          "end_class":"destination group"}'
        The field that makes reuse mechanical. Without it the recipe is a
        description; with it, a dataset can be checked against it.
    interaction: static | hover | scroll-step | click-filter | animated.
    medium: html/d3 | ggplot2 | plotly | svg | leaflet.
    code: Working code if there is any. Never required — a recipe with a
        precise contract and no code is still reusable.
    caveats: How this figure can mislead. Please fill this in; a stored
        method with no stored failure mode is a way to be wrong faster.
    default_style: viz_styles.slug to use unless told otherwise.
    derived_from: viz_exemplars.slug this came from, copied verbatim.

Returns:
    The stored recipe, plus warnings for anything left dangling.

Styles are rows here rather than prose in an agent prompt for one reason:
prose cannot be added to by saying "I like this". Before this table, the
seven styles Visualization Maker claimed were hand-edited markdown, and no
eighth was ever added.

Args:
    name: e.g. "Upshot editorial", "paper austere".
    one_liner: The feel, in one sentence.
    good_for: paper | briefing | dashboard | talk | poster.
    palette: Hex values AND what each one means. A palette whose roles are
        unrecorded gets reused wrongly.
    type_scale: Families, weights, sizes.
    axis_treatment: Grid, ticks, spines — usually how much to erase.
    annotation_rule: Direct labels vs legend, and title voice. Often the
        most transferable single rule a style has.
    motion: none | transition | scroll-driven.
    theme_pair: Light/dark token pairs, if the style has both.
    notes: Anything else worth carrying.
    derived_from: viz_exemplars.slug, copied verbatim.
    unverified: Fields written from memory rather than measured off the
        original. Exact hex values are the usual case.

Returns:
    The stored style.

This is the tool the whole library exists to serve. Eleven agent context
files were once written and never read once — a store nothing queries is
worse than no store, because it looks like progress. So: consult this before
designing a figure, not after.

Each recipe prints a **Needs** line: the roles its data contract demands. To
check a specific dataset against one, propose a role -> column mapping and
confirm it with `check_viz_fit`. This tool deliberately will not guess that
mapping — a name-similarity guess was tried here and could not tell a
fitting dataset from an unfitting one.

Args:
    kind: chart | map | diagram | flow | table | layout. Blank = all.
    intent: Free text matched against names, one-liners, encodings and
        media, e.g. "show dispersion not averages", "for a paper".
    limit: Max recipes to return.

Returns:
    Matching recipes with their contracts and track records, plus candidate
    styles ranked by how often they were loved.

The tool that makes "reproduce this with other data" a check rather than a
hunch — and it contains no cleverness on purpose. Reading that
`screening_mode` is the origin category in this dataset is semantics, which
is your job. Confirming that every role was accounted for, that each column
named actually exists, and that nothing was mapped which the recipe never
asked for is bookkeeping, which is this tool's job.

An earlier version guessed the mapping by comparing role names to column
names. It reported the same "0 matched" for a dataset built to fit and one
that could not fit at all, because roles are abstract precisely so they
carry no dataset's vocabulary. A check that cannot fail differently is
decoration, so it was removed.

Args:
    recipe: viz_recipes.slug.
    mapping: JSON object of role -> column in YOUR data, e.g.
        '{"unit_id":"health_zone","start_class":"endemicity_2015",
          "end_class":"endemicity_2025","group":"province"}'
    columns: Optional comma-separated full column list. Supply it and the
        check also catches a column you named that does not exist, plus
        reports what is left unused.

Returns:
    Ready or not-ready, what is missing, and the recipe's caveats — reprinted
    here because the moment of use is when a caveat is actually read.

The table that answers "styles that we used". Without it the library holds
intentions and no evidence, so nothing can be ranked and every session
re-argues which style to reach for.

Args:
    recipe: viz_recipes.slug.
    style: viz_styles.slug.
    dataset: What it was rendered against, in plain words.
    project_id: e.g. "angola-hat-analysis".
    output_path: Local file written.
    artifact_url: If it was published as an Artifact.
    verdict: loved | fine | wrong. Leave blank until he has actually said —
        a guessed verdict poisons the ranking it feeds.
    note: What worked, or what went wrong.

Returns:
    Confirmation plus the recipe's running track record.

Read this when the researcher asks what styles exist, or before offering a choice
of look. The same text is served as the `metis://viz-styles` resource, so
Claude Desktop sees it without any client-specific work.

Returns:
    Markdown: styles with their loved-render counts, recipes with their data
    contracts and track records.