A
licenseB
qualityF
maintenanceEnables LLMs to interact with GitHub issues by providing details as tasks, allowing for seamless integration and task management through GitHub's platform.
1
18 npm
14
MIT
triage-mcp
by architect-34
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@triage-mcpFind similar issues reporting 'XSS in markdown preview'"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
triage-mcp
Issue triage for a busy GitHub repository is repetitive human work: for each new issue, someone checks whether it duplicates an existing report and routes it to the right component. triage-mcp turns a repository's history into a local retrieval-and-classification service and exposes it to an LLM over the Model Context Protocol, so the model can answer "has this been reported before?" and "which component owns this?" from real evidence. The server itself makes no model calls — it returns retrieved issues, similarity scores and predictions, each carrying the issue numbers it was derived from, and the client's LLM does the reasoning.
Built as a one-day, local-first project: no Docker, no database server, no managed vector store. State is Parquet, a NumPy matrix and a pickled scikit-learn estimator on disk.
Every number in this README is produced by a single evaluation run and is
reproducible with make eval (seed 20260720). The committed run lives in
results/20260720T201357Z/.
Architecture
flowchart LR
GH["GitHub REST API<br/>(issues + PRs)"]
P[("data/issues.parquet<br/>47,804 issues")]
E[("embeddings.npy<br/>47,804 x 384")]
EV["evals.py<br/>time-split harness"]
R[("results/ per run<br/>metrics.json<br/>classifier.joblib")]
S["server.py<br/>FastMCP (stdio)"]
C["LLM client<br/>Claude Desktop / Code"]
U(["Grounded triage<br/>with cited issues"])
GH -->|"ingest.py: drop PRs, paginate, checkpoint"| P
P -->|"store.py: all-MiniLM-L6-v2"| E
P --> EV
E --> EV
EV -->|writes| R
P --> S
E --> S
R -->|"fitted classifier"| S
S <-->|"MCP tools: grounded JSON + scores"| C
C --> UFour stages, each its own module:
Stage | Module | What it produces |
Ingest |
| Harvests issues via the GitHub REST API, drops pull requests, paginates with per-page checkpointing, validates each record with Pydantic → |
Embed |
| Encodes |
Evaluate |
| Builds a time-split classification task, scores three methods and a retrieval proxy → |
Serve |
| Exposes six grounded tools over MCP/stdio; loads corpus, vectors and classifier lazily |
Related MCP server: GitHub Support Assistant
Quickstart
Requires Python 3.11+ and uv.
make setup # uv sync (pinned deps)
cp .env.example .env # add a GITHUB_TOKEN (raises the REST rate limit)
make ingest REPO=microsoft/vscode MAX=3000 # -> data/issues.parquet
make embed # -> data/embeddings.npy (content-hashed; re-runs are free)
make eval # -> results/<ts>/ + data/classifier.joblib
make serve # run the MCP server on stdioThe GitHub list endpoint caps at 10,000 items (~2 months for a repo this busy). The committed corpus is a full year, harvested by walking creation-date windows through the Search API:
uv run python -m triage_mcp.ingest --via search --since 2025-07-20Other targets: make stats (corpus summary), make smoke (spawn the server and
exercise every tool), make test, make lint.
Results
From results/20260720T201357Z/metrics.json.
Corpus: 47,804 microsoft/vscode issues, split by creation date into 38,243
train (2025-07-20 → 2026-04-14) and 9,561 holdout (2026-04-14 → 2026-07-20).
Component classification (holdout)
method | accuracy | macro-F1 | weighted-F1 | p50 latency | p95 latency |
majority baseline | 0.9485 | 0.1217 | 0.9235 | — | — |
TF-IDF + logistic regression | 0.7566 | 0.3070 | 0.8300 | 0.92 ms | 2.01 ms |
embedding kNN (k=10) | 0.9407 | 0.2310 | 0.9274 | 4.97 ms | 6.84 ms |
Read accuracy with suspicion here. The majority baseline scores 0.9485
accuracy by predicting other for everything — because other is 9,069 of the
9,561 holdout issues (94.8%). Its macro-F1 is 0.1217. Macro-F1 is the honest
comparison metric, and by it, TF-IDF + logistic regression (0.3070) is the
best method — more than double the majority baseline and ahead of embedding
kNN (0.2310). Latency is measured per query, one at a time, as the model would
be served; it is hardware-dependent and not seeded, unlike the quality metrics.
Per-class F1 for the winning method shows where the signal is (full table in
summary.md): chat-billing 0.494 and
accessibility 0.413 are learnable; chat 0.036 is not — vscode spreads chat
work across many chat-* labels and applies the bare chat label
inconsistently, so it is a property of the label taxonomy, not the model.
Retrieval proxy — label-match precision@5
This is a proxy, not a duplicate-detection rate. It measures how often a retrieved neighbour shares the query's component class — not whether it is actually a duplicate. This corpus has no labelled duplicate pairs, so no duplicate metric can be computed directly. Two issues in the same component are usually not duplicates. Treat this as a relative signal for comparing retrieval methods.
method | P@5 (all holdout) | P@5 (excl. | p50 latency | p95 latency |
embeddings (all-MiniLM-L6-v2, exact cosine) | 0.8746 | 0.2122 | 4.71 ms | 6.02 ms |
TF-IDF cosine | 0.8877 | 0.0959 | 160.65 ms | 203.58 ms |
The "all holdout" column is dominated by other-matching-other (noise
agreeing with noise), which is why TF-IDF looks marginally ahead there. On the
slice that means something — the 492 holdout issues that carry a real component
label — dense embeddings are 2.2× better than TF-IDF (0.2122 vs 0.0959), at
a fraction of the query latency (4.71 ms vs 160.65 ms p50). That gap is the case
for embeddings in this project.
Methodology
Time-based split. The holdout is the most recent 20% of issues by creation date; every training issue predates every holdout issue. The boundary is a timestamp, not a row index, so issues sharing the boundary instant all fall on the holdout side — no training issue is contemporaneous with a holdout one. A random split would let the model learn from the future, and on issue trackers that inflates scores badly, because label vocabulary and topics drift week to week. A leakage-guard test asserts the separation, and embedding kNN restricts its candidate pool to the training split via a boolean mask applied before similarity is computed, so a holdout issue can never retrieve itself or a future sibling.
Task construction is a judgment call, made auditable. GitHub labels mix
component/area (terminal, git), status (info-needed, duplicate), type
(bug, feature-request) and provenance (ai-generated). Only component
labels make a meaningful classification target, so the rest are excluded by an
explicit stoplist. Classes are the top-K component labels ranked on the
training split only; multi-label issues take their most frequent class;
everything else collapses to other. Classes with fewer than 10 holdout
examples are demoted to other before scoring — a class with two holdout issues
produces an F1 that swings wildly on a single prediction and would corrupt the
macro average. Here 7 of the 10 requested classes survived; chat-agents-view
(416 train / 0 holdout), chat-agent (272/4) and chat-prompts (237/9) were
demoted. The complete mapping, stoplist and demotion log are written to
class_map.json.
Three methods, weakest first.
Majority baseline — always predict the most frequent training class. The floor that exposes how misleading accuracy is on an imbalanced corpus.
TF-IDF + logistic regression — bag-of-words (1–2 grams) with
class_weight="balanced", vectoriser fit on the training split only (fitting on the full corpus would leak holdout vocabulary and IDF weights).Embedding kNN — cosine top-10 neighbours from the training split, majority vote.
Metrics are computed, never asserted. Everything comes from a run of
evals.py and is written to results/<ts>/metrics.json; nothing is hardcoded
or carried between runs. The classification quality metrics are deterministic
(seed 20260720) and reproduce bit-for-bit; latency is not. classify_component
in the server reports the classifier's measured macro-F1, not a self-assessment.
Using it from an LLM client
The server speaks MCP over stdio. It makes no LLM or network calls; every tool returns typed, structured data with the supporting issue numbers.
Tool | Returns |
| Corpus size, date coverage, embedding model, classifier provenance and measured scores |
| Ranked issues: number, title, labels, similarity, snippet |
| Candidates flagged above/below an (uncalibrated) threshold, with evidence snippets |
| Predictions from logreg and kNN, plus the neighbour issues behind the vote |
| Stored metadata for one issue |
| Duplicates + classification + every cited issue number, in one object |
Registration for Claude Desktop and Claude Code (verified against the current docs): docs/mcp-setup.md. In short, for Claude Code:
claude mcp add --scope local triage-mcp -- uv run python -m triage_mcp.serverDemo — every tool over stdio (make smoke)
The smoke test spawns the server as a subprocess and drives it through the SDK client, the same way a real client does. Abridged real output:
corpus_info
corpus_size: 47804 repos: ['microsoft/vscode']
created_from: 2025-07-20 … created_to: 2026-07-20
embedding_model: sentence-transformers/all-MiniLM-L6-v2 backend: sentence-transformers
classifier_source_run: 20260720T201357Z measured_macro_f1: 0.3069562629459519
search_similar_issues(query='terminal hangs during build', k=5)
#314312 sim=0.6586 Window hang when searching in terminal [bug, confirmation-pending, terminal-find]
#265289 sim=0.6545 Terminal is hanging [info-needed]
cited_issue_numbers: [314312, 265289, 265290, 292878, 314080]
classify_component(text=<new terminal-freeze issue>)
logreg: 'chat-terminal' p=0.8304 (runner-up 'other' p=0.1092)
kNN: 'other' vote=0.8000
methods_agree: False # the two methods disagree — surfaced, not hidden
measured_quality: macro_f1=0.3070 (from run 20260720T201357Z)
triage(text=<new terminal-freeze issue>)
duplicates above threshold: 10
predicted component (logreg): 'chat-terminal' (kNN): 'other'
cited_issue_numbers: [259318, 265622, 267001, 268344, 271668, …]
caveats: 2 attachedThat methods_agree: False is the design working as intended: the server hands
the client both predictions and the evidence, rather than manufacturing a single
confident answer the data does not support.
Limitations & future work
This is a one-day build, and it is honest about what it is not:
No duplicate-pair ground truth. The retrieval metric is a proxy. The right fix is to mine real duplicate pairs from issue timelines — vscode bots post
*duplicateand "duplicate of #N" cross-references on close — and score precision/recall/MRR against those, turningfind_duplicatesfrom a heuristic into a measured capability.No LLM reranking. Retrieval is a single dense-cosine pass. A cross-encoder or an LLM reranker over the top-k would likely lift precision on the hard cases where lexical and semantic similarity disagree.
The client drives the loop manually. A scripted agent loop that calls the tools, validates each response against its Pydantic schema, and emits a structured triage report would make the end-to-end capability testable in CI, independent of a human in a chat client.
In-memory exact search. Brute-force cosine over ~48k × 384 is milliseconds and needs no index, which is the right call at this scale. Beyond a few hundred thousand issues, moving the corpus and vectors into Postgres + pgvector would keep it a single dependency while restoring sub-linear search.
Single repository. The whole pipeline is
repo-parameterised but only vscode is ingested and evaluated. Multi-repo evaluation — does a classifier trained on one repo transfer, or is triage inherently per-repo? — is the natural next experiment.
Some of these limitations are visible in the numbers above: other is 94.8% of
the corpus because most vscode issues carry no component label at all (13,026
are entirely unlabelled; another 30,696 have only status/type labels), which
caps how high macro-F1 can realistically go on this task.
Project conventions
Type hints throughout; Pydantic for every record that crosses a boundary; unit
tests never touch the network (an autouse fixture blocks sockets, and the
server integration tests spawn a real subprocess pinned offline). See
CLAUDE.md for the full set. Dependencies are pinned in
pyproject.toml; make lint runs ruff check and
ruff format --check.
License
MIT.
Available Tools
6 toolsclassify_componentA
Predict which component/area an issue belongs to, by two methods.
Reports the logistic-regression prediction and the embedding-kNN vote
separately, along with the neighbour issues behind the vote, so you can see
whether the methods agree and what evidence supports them. measured_quality
carries the scores from the evaluation run that produced this classifier --
quote those rather than assuming the prediction is reliable.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| knn | Yes | |
| caveat | Yes | |
| logreg | Yes | |
| exemplars | Yes | The kNN neighbours that produced the vote -- the evidence behind it |
| known_classes | Yes | |
| methods_agree | Yes | |
| measured_quality | Yes | Scores from the eval run that produced this classifier, not self-reported |
| cited_issue_numbers | Yes |
TDQS
A4/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden and does it well: it discloses that two model outputs are returned separately, that neighbour issues are included as evidence, and that measured_quality comes from an evaluation run and should be quoted instead of assuming reliability. It does not mention side effects, but a prediction call is plausibly read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose and every sentence contributes: the first sentence states the task, the second explains the output and the reliability caveat. There is no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter classifier with an output schema, the description covers the main operational context: what is predicted, how the two methods are reported, and how to interpret measured_quality. It falls just short of full completeness by not addressing when to choose this over sibling tools or input text constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning to the single 'text' parameter. It implies the text is an issue and that the result is a component/area, but it does not specify input format, length, or whether the text should be title, body, or both.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and object: 'Predict which component/area an issue belongs to'. It then clarifies the method (two methods, predictions reported separately), which makes it easy to distinguish from sibling search/triage tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied: call this when you need a component/area prediction with supporting evidence. However, there is no explicit 'when to use vs alternatives' guidance, and no exclusions are given relative to sibling tools like triage or search_similar_issues.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
corpus_infoA
Describe what this server is serving: corpus coverage, model, and measured quality.
Call this before relying on the other tools -- it says what date range the corpus covers, so you can tell whether a "no duplicates found" answer means "none exist" or "none within the ingested window".
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| notes | Yes | |
| repos | Yes | |
| created_to | Yes | |
| corpus_size | Yes | |
| created_from | Yes | |
| embedding_model | Yes | |
| embedding_backend | Yes | |
| classifier_classes | Yes | |
| classifier_source_run | Yes | |
| classifier_measured_accuracy | Yes | |
| classifier_measured_macro_f1 | Yes |
TDQS
A4.7/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It transparently indicates the tool is informational ('describe what this server is serving') and explains the significance of the date range for other tools. While it doesn't explicitly state non-destructive behavior, the nature of the description implies a read-only metadata operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, consisting of two well-structured sentences. It avoids redundancy and directly conveys the tool's function and usage hint without any fluff. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that the tool has an output schema (not shown) and is a metadata endpoint, the description covers all necessary aspects: what it reports (coverage, model, quality) and why it should be called first. It also connects to the sibling tools by explaining how the date range affects interpretation, making it contextually complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, which sets the baseline at 4. There are no parameter semantics to clarify, and the description doesn't need to explain any inputs. The output schema is mentioned but not detailed, which is acceptable given the no-parameter context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: to describe the server's corpus coverage, model, and measured quality. It is specific about the resource and the aspects it covers, making it distinct from the sibling tools that search, find, classify, get, or triage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs to call this tool before relying on others, and explains why: it reveals the date range of the corpus, which is essential for interpreting 'no duplicates found' responses. This provides clear when-to-use guidance and context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
find_duplicatesA
Retrieve likely-duplicate candidates for a new issue's text.
Returns the k nearest issues, each flagged for whether it cleared the
threshold. The threshold is an uncalibrated heuristic -- see the caveat
field -- so treat flagged items as candidates to read, not as confirmed
duplicates, and cite issue numbers when reporting them.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| k | No | ||
| text | Yes | ||
| threshold | No |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| caveat | No | |
| threshold | Yes | |
| candidates | Yes | |
| n_above_threshold | Yes | |
| cited_issue_numbers | Yes |
TDQS
A4.2/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description openly discloses a key behavioral limitation: the threshold is uncalibrated, and results are not definitive. It also mentions a 'caveat' field in the output, setting expectations about the reliability and interpretation of results. This goes beyond a basic functional statement and informs the agent of potential pitfalls, which is valuable for safe usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, using two sentences to convey the purpose, the return format, and critical caveats. Every phrase adds value: the core action, the output structure, the heuristic caveat, and the usage instruction. There is no redundant or extraneous information, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the sibling tools include 'search_similar_issues' and 'classify_component,' the description adequately positions this tool as a duplicate finder with a cautionary note. It references the output 'caveat' field, tying the description to the expected result structure. However, it could be more complete by explicitly defining acceptable ranges for 'threshold' or 'k,' but the overall context is sufficient for an agent to decide when to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides no descriptions for 'text,' 'k,' or 'threshold,' so the description must compensate. It does clarify 'text' implicitly ('new issue's text') and 'k' ('k nearest issues'), but 'threshold' is only mentioned as an uncalibrated heuristic without elaborating on its value range or how it affects results. This partial explanation leaves some ambiguity, though the core parameters are understandable. A more detailed description of 'threshold' would improve the score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Retrieve likely-duplicate candidates for a new issue's text' – a specific verb ('retrieve'), a clear resource ('duplicate candidates'), and the input scope ('new issue's text'). It distinguishes itself from the sibling 'search_similar_issues' by emphasizing 'duplicates' rather than general similarity, providing immediate orientation for an agent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description offers explicit usage guidance by warning that the threshold is an 'uncalibrated heuristic' and instructing to treat flagged items as 'candidates to read, not as confirmed duplicates,' plus the directive to 'cite issue numbers when reporting them.' While it does not directly contrast with 'search_similar_issues,' the phrasing implies when this tool is appropriate (duplicate detection) versus general similarity search, which is sufficient for most agents.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_issueA
Look up one issue by number and return its stored metadata.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| number | Yes |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| url | No | |
| body | No | |
| repo | No | |
| found | Yes | |
| state | No | |
| title | No | |
| labels | No | |
| number | Yes | |
| comments | No | |
| closed_at | No | |
| created_at | No | |
| state_reason | No |
TDQS
A4.3/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavior. It states 'Look up' which implies a read-only, non-mutating operation, and 'return its stored metadata' clarifies the outcome. It does not mention error conditions or side effects, but for a simple getter, this level of transparency is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no waste. It is front-loaded with the key verb and resource, immediately conveying the tool's purpose. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a single required parameter and an output schema (as indicated by context signals), so the return value is documented. The description fully enables an agent to call the tool correctly: provide a number, get the issue's metadata. No critical information is missing for this simple, well-scoped tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds the phrase 'by number' which confirms that the 'number' parameter is the issue number, but this is already implicit from the tool name. The description provides minimal extra meaning beyond the schema, so a 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Look up' with a clear resource 'one issue by number' and states the action 'return its stored metadata'. This clearly distinguishes it from siblings like search_similar_issues, find_duplicates, and classify_component, which perform different operations. The purpose is unambiguous and directly tied to the tool name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when you have an issue number and want its metadata, providing clear context. However, it does not explicitly mention when not to use it or alternatives, such as using search_similar_issues for fuzzy matches. Since the tool is straightforward and the context is self-evident, this earns a 4.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_similar_issuesA
Find issues semantically similar to a free-text query.
Ranked by exact cosine similarity over sentence-embeddings of title + body. Use this to answer "has anything like this been reported?" -- then cite the returned issue numbers.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| k | No | ||
| query | Yes |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| k | Yes | |
| hits | Yes | |
| query | Yes | |
| corpus_size | Yes | |
| cited_issue_numbers | Yes |
TDQS
A4.5/5.0
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the ranking mechanism ('exact cosine similarity over sentence-embeddings') and implies read-only behavior (searching issues). There are no hidden side effects or destructive actions. Since no annotations are provided, the description fully carries the transparency burden and does so effectively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (two sentences) and front-loaded with the primary purpose. It then provides implementation detail and a usage example. Every sentence adds value, with no fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers what the tool does, how it ranks results, and when to use it, including a concrete example ('cite the returned issue numbers'). It is self-contained for an agent to decide and invoke appropriately, despite not specifying the return format, which is not essential for a search tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description does not mention parameters 'query' or 'k,' but the input schema already fully defines them with types and a default for 'k.' Since schema coverage appears complete, the description adds no additional meaning beyond the schema, warranting a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: finding issues semantically similar to a free-text query. It uses the specific verb 'Find' and resource 'issues,' making it unambiguous. It also implies differentiation from sibling tools by focusing on semantic similarity rather than other operations like duplicate detection or triage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a direct use case: 'Use this to answer "has anything like this been reported?"' This gives clear guidance on when to apply the tool. It does not explicitly contrast with sibling tools, but the stated scenario is specific enough to guide appropriate usage. A minor gap is the lack of explicit 'when not to use.'
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
triageA
Full triage pass over new issue text: duplicates plus component routing.
One call combining find_duplicates and classify_component, with every
supporting issue number collected into cited_issue_numbers. Narrate this
for the user -- including the caveats -- rather than presenting it as a
decision already made.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| caveats | Yes | |
| duplicates | Yes | |
| text_preview | Yes | |
| classification | Yes | |
| cited_issue_numbers | Yes |
TDQS
A4.4/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses key behaviors: it combines two operations, collects supporting issue numbers into cited_issue_numbers, and frames output as narration with caveats. No annotation exists, so this description carries the burden adequately.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, with no redundant or irrelevant content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers what the tool does, how it behaves, and how the result should be presented. Minor gap: output structure is not described, but the narrative guidance compensates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, text, is contextually described as 'new issue text' in the description, making its purpose clear even though the schema itself lacks a description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool performs a full triage pass combining duplicate detection and component routing, which distinguishes it from sibling tools focused on individual subtasks.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains that this is a single-call combination of find_duplicates and classify_component, implicitly indicating when to use it over the individual tools; also provides guidance on narrating caveats rather than presenting decisions as final.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
6 tool updates
v0.1.0- First observed
classify_component - First observed
corpus_info - First observed
find_duplicates - First observed
get_issue - First observed
search_similar_issues - First observed
triage
TDQS
A4.1/5.0
Scored across 6 tools
Disambiguation4/5
Most tools have clearly distinct purposes: search_similar_issues is general semantic search while find_duplicates is duplicate-specific, and triage explicitly bundles two other tools. The only real overlap is between search_similar_issues and find_duplicates, but their descriptions clarify the intended use case.
Naming Consistency3/5
Most tools follow a verb_noun pattern (search_similar_issues, find_duplicates, classify_component, get_issue), but 'triage' is a bare verb and 'corpus_info' is a noun_noun pair, breaking the otherwise consistent convention. The naming is still readable and predictable overall.
Tool Count5/5
Six tools is well-scoped for an issue-triage server: semantic search, duplicate detection, component classification, single-issue lookup, a combined triage pass, and corpus metadata. Each tool has a distinct role and none feel redundant.
Completeness4/5
The server covers the core triage workflow end-to-end: search, duplicate identification, component routing, and full triage, with corpus_info to contextualize results. A minor gap is that triage only accepts raw text rather than an issue number, though agents can work around this with get_issue.
Maintenance
ActivitySlowing
ResponsivenessNo issues
Related MCP Connectors
Code intelligence for LLMs. Analyze, search, and retrieve code from any public git repository.
Finds real, maintained open-source repos that fit your project. MCP grounding for coding agents.
Code intelligence for coding agents: semantic, AST, graph, and full-text search. 279+ languages.
Connect AI assistants to your GitHub-hosted Obsidian vault to seamlessly access, search, and analy…
Related MCP Servers
- AlicenseBqualityNot gradedmaintenanceHelps support engineers find similar GitHub issues to speed up troubleshooting by searching repositories and calculating similarity scores based on issue descriptions.12-
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to search and retrieve context from GitHub issues, pull requests, releases, and documentation using hybrid semantic search and time-ordered activity scans.108 npmApache 2.0
- FlicenseNot gradedqualityDmaintenanceEnables searching and retrieving knowledge extracted from GitHub PR reviews to help developers avoid repeating mistakes.-