Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| GITHUB_TOKEN | No | GitHub token used for authentication. Note that gh CLI usually manages its own auth; this is typically only set for CI environments. |
Capabilities
Features and capabilities supported by this server
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| list_submissions | Fetch the list of Agents League submissions. Args:
track: Track name to filter by.
``"creative-apps"`` | ``"reasoning-agents"`` | ``"enterprise-agents"`` | None (all)
state: Issue state. ``"open"`` | ``"closed"`` | ``"all"``
Returns:
A list of submission summaries. Each element is a dictionary
containing issue_number, title, track, project_name, repo_url,
created_at, has_demo.
Raises:
RuntimeError: When gh command execution fails. |
| get_submission_detail | Fetch detailed submission data for the specified Issue number. Parses each section of the Issue template and returns scoring data.
GitHub Username is hidden during scoring to eliminate bias, but
retained as the github_username field for report output.
If repo_url points to a GitHub repository, the README is also fetched.
Args:
issue_number: The Issue number to fetch.
Returns:
A dictionary containing detailed submission information.
Raises:
RuntimeError: When gh command execution fails. |
| get_scoring_rubric | Return the scoring rubric for the specified track. Loads the YAML file ``data/rubrics/{track}.yaml`` and returns
the scoring criteria (name, weight, description, scoring_guide).
Args:
track: Track name. ``"creative-apps"`` | ``"reasoning-agents"``
| ``"enterprise-agents"``
Returns:
Rubric dict with track, track_display_name, criteria (list),
total_weight, score_range, and notes.
Raises:
FileNotFoundError: If the YAML file for the track does not exist.
ValueError: If the track name is invalid. |
| save_scores | Save scoring results to data/scores.json. Existing scores for the same Issue are overwritten (idempotent).
New Issues are appended.
Args:
scores: List of scoring result dicts. Each must contain:
- issue_number (int)
- project_name (str)
- track (str)
- criteria_scores (dict[str, int]): per-criterion scores (1-10)
- weighted_total (float): weighted total (0-100)
- evidence (dict[str, str]): per-criterion evidence citations
- confidence (str): 'high', 'medium', or 'low'
- red_flags_detected (list[str]): red flag signals found
- bonus_signals_detected (list[str]): bonus signals found
- strengths (list[str])
- improvements (list[str])
- summary (str)
Returns:
Summary dict (saved_count, updated_count, total_in_store, file_path).
Raises:
OSError: If disk write fails. |
| generate_ranking_report | Generate a Markdown ranking report and save to reports/ranking.md. Reads scoring results from data/scores.json and produces a report
containing overall ranking, per-track ranking, and individual
evaluation summaries.
Args:
top_n: Number of top entries to highlight (default: 10).
Returns:
Result dict (report_path, total_scored, top_n, top_entries). |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |