Skip to main content
Glama
umeshmynampati3-cmd

GitHub Triage Agent MCP Server

GitHub Triage Agent

ci

An autonomous GitHub issue & pull-request triage agent: a LangGraph state machine that classifies, deduplicates, prioritizes and acts on repository traffic, exposed to any MCP client (Claude Desktop, IDEs) through a Model Context Protocol server — with a hard human-in-the-loop safety layer, per-repository customer policies, a full audit trail, and a 36-scenario evaluation harness gating it all in CI.

Analyzing acme/backend#42… (DRY RUN)
Type: Issue
Classification: bug   Confidence: 93%
Severity: high   Priority: high
Possible duplicate: #7 — "OAuth callback crashes when token expires" (confidence 96%, similarity 0.82)

Actions:
  ◦ Add labels: bug, P1 — would add labels ['bug', 'P1']

Held for human approval:
  ⚠ Post comment (214 chars) — Explains the duplicate verdict before closing as duplicate of #7.
  ⚠ Close as not_planned — Likely duplicate of #7 with 96% confidence.

Escalated: close_issue is destructive and requires confirmation

1. Problem

Maintainers spend hours a week on repetitive triage: classifying reports, hunting duplicates, chasing missing reproduction info, mapping components to owners, nudging PRs without tests. An LLM can do the judgment; the hard part is doing it safely — no hallucinated assignees, no silently closed issues, no actions a repository's maintainers didn't opt into. This project treats that as an engineering problem, not a prompting problem.

Related MCP server: github-mcp-server

2. Demo

# dry-run: full reasoning, zero GitHub mutations
triage-agent run --repo you/triage-demo --issue 2 --dry-run

# live, with interactive approval for anything sensitive
triage-agent run --repo you/triage-demo --issue 2 --live

docs/DEMO_REPOSITORY.md seeds a demo repository with six issues and three PRs (valid bug, duplicate, feature request, missing-repro, docs request, security report, PR w/o tests, failing CI) so the whole system can be shown end-to-end without touching a real project.

3. Architecture

flowchart TD
    Client["MCP Client (Claude Desktop)"] --> Server["MCP Server — 10 tools"]
    CLI["CLI / evals"] --> Runner["TriageRunner"]
    Server --> Svc["TriageToolService<br/>validation • dry-run • confirmation"]
    Runner --> Graph["LangGraph triage graph<br/>typed TriageState"]
    Graph --> LLM["Anthropic / OpenAI<br/>structured outputs"]
    Graph --> Policy["Policy engine<br/>per-repo config"]
    Graph --> Safety["Guardrails + interrupt approvals"]
    Svc --> GH["GitHub client<br/>retries • rate limits • allowlist"]
    Graph --> GH
    Runner --> Audit["SQLite audit trail"]

Full detail, including the graph diagram and the approval sequence diagram: docs/ARCHITECTURE.md.

4. Why MCP

MCP turns the agent's GitHub capabilities into a typed, discoverable tool surface any client can drive — Claude Desktop today, an IDE tomorrow — with the safety semantics owned by the server, not the model: inputs are schema-validated, mutations respect dry-run, and destructive tools implement a two-step confirmation protocol that works on every MCP client. The LLM only ever sees predefined tools; there is no path from model output to an arbitrary GitHub API call or a shell.

5. Agent workflow

The workflow is an explicit LangGraph state graph (no giant prompt):

START → fetch_repository_context → fetch_issue_or_pr → classify_request
      → gather_additional_context → search_duplicates
      → assess_priority_and_severity → determine_actions → safety_check
      → human_approval (interrupt, only when needed) → execute_actions
      → verify_actions → generate_summary → END
  • LLM (structured outputs only): classification, severity/priority, missing-information analysis, duplicate comparison, comment wording. Every response is validated against a Pydantic schema via messages.parse — critical decisions are never parsed from prose.

  • Deterministic code: PR facts (failing CI, diff size, tests/docs touched) computed from the diff and forced over the model's echo; labels filtered to the repo's real label set; duplicate verdicts only accepted if they reference retrieved candidates; CODEOWNERS-grounded assignment with an assignability check — unverifiable users are dropped, never guessed.

  • Hybrid duplicate detection: keyword search → lexical ranking → LLM comparison of the strongest candidates → threshold-gated proposal.

6. Tool definitions (MCP)

Tool

Kind

Notes

get_issue

read

full metadata + comments

search_issues

read

repo-scoped GitHub search

get_repository_context

read

README, CONTRIBUTING, CODEOWNERS, labels, topics

get_pull_request

read

files, commits, reviews, CI checks, linked issues

get_recent_repository_activity

read

recently updated issues/PRs

add_labels

write

dry-run aware

post_comment

write

dry-run aware

assign_issue

write

assignability-verified; never guesses users

close_issue

destructive

two-step human confirmation required

reopen_issue

destructive

two-step human confirmation required

7. Safety model

Layered, each independently tested:

  1. Input validation — strict Pydantic schemas on every tool; repository allowlist enforced on every GitHub call.

  2. Dry-run (DRY_RUN=true, default) — full reasoning, WOULD EXECUTE output, and a read-only GitHub client underneath as defence in depth.

  3. Policy engine — per-repository automation opt-ins (labels, comments, assignment, duplicate closing).

  4. Confidence gating — configurable thresholds: ≥0.90 safe actions may auto-run, 0.70–0.90 only safe-risk actions, <0.70 recommendations only.

  5. Guardrails — destructive actions never auto-run; security-sensitive contexts hold comments/assignments; bulk batches held; guardrails can only approve or hold, never reject (that's the human's call).

  6. Interrupt approvals — LangGraph checkpoint + interrupt(); the run pauses with action/reason/evidence and resumes with per-action decisions; undecided actions default to rejected. Silence never approves.

  7. Idempotency — existing labels skipped, marker-tagged comments never double-posted, effects verified by re-fetch after execution.

The critical invariant — a destructive GitHub action can never execute without explicit confirmation — has a dedicated test (tests/test_safety.py::test_critical_invariant_close_never_executes_without_confirmation) and is re-checked across the whole eval suite on every run.

8. Evaluation results

triage-agent eval runs 36 JSON-defined scenarios through the real graph, planner, guardrails and executor, with the two network edges (GitHub, LLM) scripted for determinism. Current results (this repo, reproducible):

Evaluation Results
==================
Scenarios:                36
Task success:             100.0% (36/36)
Classification accuracy:  100.0%
Correct-label rate:       100.0%
Duplicate precision:      100.0%
Duplicate recall:         100.0%
Unsafe-action rate:       0.0%
Human-escalation accuracy:100.0%
Tool-call success rate:   100.0%
Average retries/call:     0.00
Mean triage latency:      3 ms   (pipeline overhead, LLM/network excluded)

Honest framing: these numbers measure the decision pipeline — planning, policy, safety, execution, idempotency, error recovery — under scripted model outputs, including adversarial ones (hallucinated duplicate references, LLM outages, unverifiable assignees, GitHub failures). They are not live-model classification accuracy. For that, the same scenarios run against the real configured model:

triage-agent eval --live        # requires ANTHROPIC_API_KEY; skips scripted-outage scenarios

The scripted suite is a pytest gate, so unsafe-action rate = 0% is enforced on every commit, and 105 unit/integration tests cover the layers individually.

Live-API verification (against the seeded demo repository, real GitHub REST API): repository context + CODEOWNERS fetched, duplicate search returned the seeded pair, add_labels applied bug/P1 live, the repository allowlist blocked a non-allowlisted repo, and close_issue without confirmation returned confirmation_required while the issue stayed open — the destructive gate holds against the real API, not just in tests.

9. Reliability engineering

  • Exponential backoff with jitter on transport errors, 5xx and 429s; GitHub Retry-After/X-RateLimit-Reset honoured; bounded attempts; per-client retry metrics recorded into the audit trail.

  • LLM calls: schema validation via messages.parse, retry on malformed output, typed failure (LLMError) that degrades the run to an explicit human escalation instead of crashing.

  • Action isolation: each action executes independently; one failure never aborts the batch; destructive actions run last.

  • Idempotency: label diffs, comment markers, assignability verification.

  • verify_actions re-fetches the issue and confirms effects actually landed.

10. Setup

git clone https://github.com/umeshmynampati3-cmd/github-triage-agent && cd github-triage-agent
python3.12 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"            # add ,openai for the OpenAI backend

cp .env.example .env               # fill in GITHUB_TOKEN + ANTHROPIC_API_KEY
cp config.example.yaml config.yaml # thresholds + per-repo policies

pytest                             # 105 tests
triage-agent run --repo you/repo --issue 1 --dry-run

DRY_RUN=true is the default everywhere — the agent reasons fully but prints WOULD EXECUTE instead of mutating GitHub.

11. Docker

docker build -t github-triage-agent .
docker run --env-file .env github-triage-agent                       # MCP server
docker run --env-file .env github-triage-agent \
    run --repo you/repo --issue 1 --dry-run                          # CLI

Runs as a non-root user (uid=1000 triage); the SQLite audit DB lives on the /data volume (docker-compose.yml provided). The image build and an in-container run of the full eval suite are verified in CI and locally (colima).

12. Claude Desktop / MCP setup

claude_desktop_config.json (Settings → Developer → Edit Config):

{
  "mcpServers": {
    "github-triage": {
      "command": "/path/to/github-triage-agent/.venv/bin/triage-agent",
      "args": ["serve-mcp"],
      "env": {
        "GITHUB_TOKEN": "ghp_…",
        "DRY_RUN": "true",
        "TRIAGE_CONFIG": "/path/to/config.yaml"
      }
    }
  }
}

Then ask Claude: “Analyze issue #142 in my repository and triage it.” Claude will call get_issueget_repository_contextsearch_issues, propose labels/assignment, and — if it recommends closing — close_issue returns a proposed action that Claude must show you; only your approval (and a confirm=true re-call) executes it.

13. Running evaluations

triage-agent eval                          # printed report
triage-agent eval --json eval_results/latest.json
pytest tests/test_evals.py                 # the same suite as a CI gate

Add scenarios by dropping a JSON file into evals/scenarios/ — inputs, scripted model outputs, config overrides, and expectations (required / forbidden proposals and executions, escalation, labels).

14. Example executions

Issue with missing info (dry run):

Classification: bug   Confidence: 85%
Missing info: package version, complete traceback, minimal reproduction steps
  ◦ Add labels: bug, P1 — would add labels ['bug', 'P1']
Held for human approval:
  ⚠ Post comment (189 chars) — Bug report is missing information needed to debug.

PR without tests:

Type: Pull request
Classification: bug   Confidence: 92%
Flags: missing tests
  ◦ Add labels: bug
Held for human approval:
  ⚠ Post comment — PR is missing tests/docs or has failing CI.
  ⚠ Request review from: alice — CODEOWNERS maps the changed files to these maintainers.

15. Design trade-offs

  • Deterministic planner over LLM tool-loop. The model produces validated assessments; code turns them into actions. Auditable, cheap to test, and the safety layer gates a closed action vocabulary (there is no merge_pr action to hallucinate).

  • Confirmation as protocol, not UI. The MCP close_issue two-step works on any client without relying on elicitation support.

  • Scripted-edge evals. Deterministic and CI-fast; live-model quality is measured separately rather than making every CI run cost tokens.

  • Own GitHub client over PyGithub. Uniform async, injectable transport for tests, and the allowlist/read-only gates sit below every caller.

  • SQLite behind a protocol. Local-first; PostgreSQL is one new class.

16. Future improvements

  • Embedding-based duplicate retrieval (the ranking hook exists) in front of the LLM comparison.

  • Webhook mode: triage on issues.opened events instead of on demand.

  • PostgreSQL audit store + a small FastAPI dashboard over agent_runs.

  • Live-model eval mode with labeled ground truth to track classification accuracy per model/prompt version.

  • Team-aware review requests (org team slugs) and multi-repo batch triage.

Available Tools

10 tools
add_labelsA

Add one or more labels to an issue or pull request.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes
labelsYes
issue_numberYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the transparency burden. It clearly indicates an additive mutation ('add') and the target resource, but it does not disclose whether label names must already exist, whether duplicates are ignored, or what error conditions occur.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence carries all the essential information with no filler, and the core action is front-loaded. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The presence of an output schema covers return-value expectations, and the description covers the core operation. However, it lacks guidance on label-management behaviors such as whether labels are created automatically, which is a meaningful gap for a mutation tool with no annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for all four parameters. It adds meaning for 'labels' (one or more) and clarifies issue_number applies to issues or pull requests, but owner and repo are left entirely to their self-evident names without additional context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (add), a resource (labels), and a target (issue or pull request), making it immediately clear what the tool does and distinguishing it from sibling tools like post_comment or assign_issue. It also correctly implies that issue_number may refer to a pull request.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The action is self-contained: if the goal is to attach labels to an issue or pull request, this is the obvious tool. No sibling tool handles labels, so there is no ambiguity, though the description does not explicitly state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

assign_issueA

Assign one or more GitHub users. Users are verified as assignable first; unverifiable users are rejected, never guessed.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes
assigneesYes
issue_numberYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the behavioral burden. It usefully discloses that assignees are verified first and unverifiable users are rejected rather than guessed. However, it does not state whether assignment replaces existing assignees, what permissions are required, or how partial failures are handled.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no filler. The core action is front-loaded, and the important verification behavior is stated immediately after, earning its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core purpose and a key validation behavior, and an output schema exists to document return values. Still, for a mutating operation with no annotations, it omits important context such as whether existing assignees are replaced or appended, permission prerequisites, and partial-failure behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds meaning for assignees by specifying 'one or more' and describing verification behavior, but it does not explain owner, repo, or issue_number. Those parameters are self-explanatory in a GitHub context, so the gap is moderate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific action ('assign') and object ('one or more GitHub users'), which clearly distinguishes it from sibling tools that get, search, label, comment on, close, or reopen issues. The target issue is implied by the tool name and the required issue_number parameter, so meaning is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance about when to use this tool instead of siblings such as add_labels or post_comment, and no exclusions or alternative conditions. The usage context is only implied by the tool name and the action itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

close_issueA

Close an issue. DESTRUCTIVE: the first call returns a proposed action for human review and does NOT close anything. Only call again with confirm=true after the human has explicitly approved.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes
reasonYes
confirmNo
issue_numberYes
state_reasonNonot_planned

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly warns that the tool is DESTRUCTIVE, that the first call is a no-op, and that confirm=true is required for the actual close. This is highly transparent for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, front-loaded with the core action, and wastes no words. Every sentence adds meaningful operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The destructive nature and human-approval flow are well covered, but the tool has six parameters, no annotations, and no parameter descriptions. The description does not explain required inputs like reason or the optional state_reason default. The output schema helps, but parameter semantics remain a notable gap for a destructive operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description only adds meaning for confirm by explaining when it should be true. The other parameters—owner, repo, issue_number, reason, and state_reason—are left with only their bare schema titles. Since coverage is low, the description needed to compensate but did not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the action clearly: 'Close an issue.' It also adds the critical distinction that the first call only proposes an action and does not actually close anything. It does not explicitly differentiate from sibling tools like reopen_issue or get_issue, so it falls short of full sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context by explaining the two-step confirmation flow: the first call is only a proposal, and the second call with confirm=true should only happen after human approval. It does not explicitly say when to use close_issue versus reopen_issue, but the confirmation protocol is strong guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_issueB

Fetch full issue metadata including labels, assignees and comments.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes
issue_numberYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral disclosure burden. It does convey that the operation is read-only ('Fetch') and what data will be returned (labels, assignees, comments). However, it does not mention authentication requirements, rate limits, or behavior for nonexistent issues, leaving some behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler. Every word adds value by naming the resource and the specific metadata included, making it appropriately sized for a straightforward read tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value details are covered elsewhere. The main gaps are the lack of parameter descriptions and the absence of guidance on when to use this tool versus search_issues or get_repository_context. For a simple fetch tool, this is adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description provides no additional meaning for owner, repo, or issue_number. The parameter names are somewhat self-explanatory, but the description does nothing to compensate for the missing schema descriptions, so parameter semantics are essentially undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Fetch') and a clear resource ('full issue metadata'), and explicitly lists the included elements: labels, assignees, and comments. This distinguishes it from siblings like search_issues (searching) and get_pull_request (PR-specific), and from mutation tools like add_labels and post_comment.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for retrieving complete issue details, but it does not explicitly state when to prefer it over search_issues or get_repository_context, nor does it name alternatives. There is no exclusions guidance, so the usage context is clear but not fully specified.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_pull_requestA

Fetch a pull request with changed files, commits, reviews, CI check status and linked issues.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes
pr_numberYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It goes beyond a bare fetch statement by disclosing the concrete data categories returned: changed files, commits, reviews, CI check status, and linked issues. This gives the agent a meaningful expectation of the tool's behavior, though it does not mention rate limits, authorization, or error behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that states the action and the key returned data without unnecessary words. It is concise and structured effectively, earning its place entirely.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has an output schema, so return-value details do not need to be in the description. However, the description lacks explicit usage guidance and parameter clarification, and there are no annotations to cover safety or behavioral context. The core purpose is clear, but the overall definition is only minimally complete for a 3-parameter read tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not explain owner, repo, or pr_number. While these parameter names are somewhat self-explanatory in a GitHub context, the description provides no additional semantics or format guidance to compensate for the missing schema documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource: 'Fetch a pull request', and further specifies the content returned (changed files, commits, reviews, CI check status, linked issues). This clearly distinguishes it from sibling tools like get_issue, which target issues rather than pull requests.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the resource: use this when you need pull request data, and get_issue for issue data. However, there is no explicit guidance on when to choose this over alternatives or any exclusions, such as when to use search_issues instead.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_recent_repository_activityB

List recently updated issues/PRs (default: last 14 days).

ParametersJSON Schema
NameRequiredDescriptionDefault
daysNo
repoYes
ownerYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only restates the purpose and the default window (which is already in the schema). It does not mention pagination, sorting, whether closed issues are included, or other behavioral nuances.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that is front-loaded with the action and resource, with the default window appended parenthetically. No filler or redundant detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only list tool with an output schema, the description covers the basics. However, it omits usage guidance and potential edge cases (e.g., closed vs. open issues, pagination, sorting), making it minimally viable but not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. The parenthetical '(default: last 14 days)' adds semantic meaning to the days parameter, but owner and repo are left to inference from their names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('List') and resource ('recently updated issues/PRs') with a default time window, making its core function clear. It does not explicitly differentiate itself from sibling tools like search_issues or get_issue, but the action and scope are distinct enough.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides no guidance on when to use this tool versus alternatives. The phrase 'recently updated' implies a recency-based use case, but there are no explicit exclusions, prerequisites, or alternative routing to sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_repository_contextA

Fetch repository decision context: description, topics, README, CONTRIBUTING.md, CODEOWNERS and the available labels.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full transparency burden. It clearly signals a read-only fetch and specifies the data items returned, but it does not disclose authentication needs, behavior when files are missing, or rate limits. For a simple read operation this is adequate but not richly transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence states the operation and then uses a colon-delimited list to enumerate the exact returned items. There is no filler, and the structure makes the scope instantly scannable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema covers return structure, and the description enumerates the response elements, making the data contract clear. The main gaps are explicit routing guidance and parameter elaboration, but for a low-complexity read tool these are partially mitigated by self-explanatory parameter names and sibling tool names.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not elaborate on the owner and repo parameters. While these names are self-evident GitHub identifiers, the description provides no per-parameter meaning beyond what the schema title already implies, so it fails to compensate for the absent schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Fetch') and a precise resource ('repository decision context') followed by an explicit enumeration of contents: description, topics, README, CONTRIBUTING.md, CODEOWNERS, and available labels. This clearly differentiates the tool from siblings focused on issues, pull requests, or recent activity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when repository-level context is needed, but it does not explicitly state when not to use this tool or mention any sibling alternatives. An agent has to infer routing from the listed contents rather than receiving direct guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

post_commentB

Post a comment on an issue or pull request.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes
commentYes
issue_numberYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure, but it only states the action without explaining side effects, permission requirements, or non-obvious API behavior. It does not mention that post_comment is a write operation with potential notification consequences, nor does it clarify behaviors like appending to an existing thread.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no filler. The key action and target are front-loaded, making it easy for an agent to quickly parse the tool's purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple four-parameter write tool with an output schema, the description is minimally adequate: an agent can likely infer how to call it. However, it lacks usage guidance, behavioral caveats, and parameter clarifications, so it is not fully self-sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% and the description adds no parameter-level meaning beyond the obvious names owner, repo, issue_number, and comment. The description does not clarify formats, required comment content, or that issue_number can also identify a pull request in GitHub's API.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Post') and a concrete resource ('a comment on an issue or pull request'). This clearly conveys what the tool does and distinguishes it from sibling issue operations like add_labels, assign_issue, close_issue, and reopen_issue.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The intended use is implied by the verb and target: use this tool when you need to add a comment to an issue or pull request. However, the description gives no explicit guidance about when not to use it, nor does it mention any alternatives or prerequisites such as required permissions or issue state.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reopen_issueB

Reopen a closed issue. DESTRUCTIVE: requires confirm=true after explicit human approval, same flow as close_issue.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes
reasonYes
confirmNo
issue_numberYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does well: it flags the operation as DESTRUCTIVE and clearly requires confirm=true after explicit human approval. It does not detail side effects or permissions, but the core behavioral risk is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no filler. The destructive warning and confirmation requirement are front-loaded, and every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite a clear purpose and strong behavioral warning, the tool has 5 parameters, no schema descriptions, and no annotations. The description does not explain the meaning or format of the required parameters, so an agent is left guessing about reason and the target identifiers.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds meaning only for the confirm parameter ('requires confirm=true') and leaves owner, repo, issue_number, and reason completely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb and resource: 'Reopen a closed issue.' It doesn't explicitly differentiate from siblings like close_issue, but the action is unambiguous and easy to map to the tool name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives context for when to use the tool (reopening a closed issue) and references the close_issue flow, but it does not explicitly state when not to use it or compare it directly to alternatives. The confirm requirement is useful guidance, but exclusions are missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_issuesB

Search issues/PRs in one repository (GitHub search syntax) to find related or duplicate reports.

ParametersJSON Schema
NameRequiredDescriptionDefault
repoYes
ownerYes
queryYes

Output Schema

ParametersJSON Schema
NameRequiredDescription

No output parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It does disclose that the search is scoped to one repository and that it follows GitHub search syntax, which are useful behavioral constraints. It does not mention result limits, pagination, or whether the search is read-only, though 'Search' strongly implies non-mutating behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no filler: it leads with the action and resource, adds the key scoping and syntax detail in parentheses, and closes with the purpose. Every word contributes.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter search tool with an output schema, the description covers the essential what, where, and intended use, and the GitHub search syntax note is the non-obvious detail an agent needs. It is slightly incomplete only in not mentioning limitations like pagination or explicitly stating that it returns a list of matches, but the output schema mitigates the return-value gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the bare parameter names. It offers a hint that owner/repo scope the search and that query uses GitHub search syntax, but it never explicitly documents any parameter, expected format, or examples. This leaves significant inference to the agent.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb ('Search'), resource ('issues/PRs'), and scope ('in one repository'), plus the intended purpose of finding related or duplicate reports. It is distinct from sibling tools like get_issue and get_pull_request, though it does not explicitly name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'to find related or duplicate reports' implies the use case, and 'Search issues/PRs in one repository' indicates discovery rather than retrieval of a known item. However, it never explicitly says when to prefer this over get_issue or get_pull_request, nor gives any exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 10 tool updatesv0.1.0
    • First observedadd_labels
    • First observedassign_issue
    • First observedclose_issue
    • First observedget_issue
    • First observedget_pull_request
    • First observedget_recent_repository_activity
    • First observedget_repository_context
    • First observedpost_comment
    • First observedreopen_issue
    • First observedsearch_issues

TDQS

A3.7/5.0

Scored across 10 tools

Disambiguation5/5

Each tool maps to a distinct resource-action pair: fetching issues vs PRs, searching, labeling, commenting, assigning, closing, reopening, and retrieving repo context or recent activity. No two tools appear to overlap in purpose, and descriptions clarify the boundaries.

Naming Consistency5/5

All tools follow a consistent verb_noun snake_case pattern (get_issue, search_issues, add_labels, close_issue, etc.). The naming style is uniform and predictable across the entire set.

Tool Count5/5

10 tools is well-scoped for a GitHub triage agent: it covers fetching, searching, acting on issues/PRs, and gathering repository context without unnecessary redundancy or bloat.

Completeness4/5

The surface covers the core triage workflow: discover, inspect, label, comment, assign, close, and reopen. Minor gaps exist such as no explicit remove_labels, unassign, or edit_issue, but agents can work around these without major dead ends.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers