Skip to main content
Glama
hzy9981

io.github.hzy9981/aviation-accident-analysis

by hzy9981
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

航空事故致因抽取与知识图谱项目 (Aviation Accident Analysis)

本项目基于 Microsoft GraphRAG 框架 ，旨在自动从航空事故报告中抽取实体与因果关系，并构建符合“人-机-环-管”框架的知识图谱。

项目结构

  • data/: 原始数据与地标数据。

  • input/: GraphRAG 输入目录（清洗后的文本）。

  • output/: GraphRAG 输出结果（Parquet, GraphML 等）。

  • prompts/: 自定义的 GraphRAG 提取提示词。

  • src/:

    • ingest.py: PDF/DOCX 解析与文本清洗。

    • evaluate.py: 提取准确率与 F1 分数评测脚本。

  • run.sh: 一键运行脚本。

  • settings.yaml: GraphRAG 配置文件。

Related MCP server: docsearch

环境要求

  • Python 3.10+

  • uv (推荐)

环境配置注意事项

  • 若遇到 LLM 连接错误，请检查 settings.yaml 中的 api_base 是否配置正确。在某些 Docker 或受限环境中，建议将 localhost 修改为 127.0.0.1。

  • 本项目已对 prompts/extract_graph.txt 中的抽取示例进行了优化，通过引入复杂场景示例，提升了在航空事故报告中的实体抽取表现。

Prompt 优化与评估记录 (2026-07-24)

优化策略

针对航空事故报告的复杂性，对 prompts/extract_graph.txt 进行了多轮手动迭代优化：

  1. 实体原子化：强制拆分复杂短语（如“燃油管路破裂”拆分为实体“燃油管路”与关系“破裂导致”），提升知识图谱的结构化程度。

  2. 严格去噪：建立了否定约束，排除通用职衔（如单独的“机长”）和指示代词（如“该部件”），提升精确率。

  3. 少样本增强：集成了 few_shot_data.csv 中的高质量航空事故致因链示例，引导模型识别长程因果关系。

  4. 因果链强制构建：明确 [因素] -> [事件] -> [结果] 的链条要求。

评估结果 (Fuzzy Match)

通过 scripts/run_automated_evaluation.py 运行自动化评估，对比 data/ground_truth.json 得到以下指标：

指标

实体 (Entities)

关系 (Relationships)

精确率 (Precision)

0.4348

0.0556

召回率 (Recall)

0.7143

0.0909

F1 分数 (F1 Score)

0.5405

0.0690

结论：实体抽取的精确率和 F1 分数相比基准版本有显著提升。关系抽取受限于本地模型的指令遵循能力，仍有优化空间。

MCP 服务 (Model Context Protocol)

本项目提供了一个 MCP 服务，方便通过 MCP 客户端（如 Claude Desktop）直接调用文档解析与提示词生成功能。

核心功能

  • 文档解析：支持 PDF、DOCX、TXT 格式的自动解析与清洗。

  • 提示词注入：将解析出的文档内容自动注入到指定的提示词模板中。

启动方式

uv run mcp_server.py

可用工具

  1. upload_and_generate_prompt:

    • 参数 file_path: 文档路径。

    • 参数 template_name: prompts/ 目录下的模板名（默认 extract_entities）。

  2. list_available_templates: 列出所有可用的提示词模板。

🐳 一键启动 (Docker)

如果你想快速启动整个分析环境（包含 Neo4j 数据库和 MCP 服务），可以使用 Docker Compose：

docker-compose up -d

启动后：

  • Neo4j 控制台: http://localhost:7474 (默认账号: neo4j, 密码: password)

  • MCP 服务: 已在容器内运行，并自动连接到容器内的 Neo4j。

运行流程

  1. 将事故报告（PDF/DOCX/TXT）放入 data/input_reports。

  2. 运行流水线：

    bash run.sh

使用 Ollama (本地 LLM)

在 settings.yaml 中修改 completion_models 和 embedding_models：

completion_models:
  default_completion_model:
    model_provider: openai
    api_base: http://localhost:6006/v1
    model: llama3 # 或其他模型
    api_key: ollama # 占位符

注意：GraphRAG 对本地模型的支持取决于模型的遵循指令能力。

提取 Schema (人-机-环-管)

  • 实体类型: person (人), equipment (机), environment (环), organization (管), procedure, event, weather, violation.

  • 关系类型: leads to, limited by, violates, not executed, belongs to.

Neo4j 知识图谱导入

  • 本项目支持将 GraphRAG 提取的实体与关系导入 Neo4j 图数据库。

  • 使用 src/export_to_neo4j.py 脚本完成导入。

  • 已配置 Docker 容器运行 Neo4j 实例，确保能够正常连接。

Available Tools

3 tools
list_available_reportsB

List all available reports in a specified directory.

ParametersJSON Schema
NameRequiredDescriptionDefault
directoryNoinput

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'list,' which implies a read operation, but it does not mention permissions, what 'available' means, whether the search is recursive, or any side effects. The description adds no behavioral context beyond the core verb.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that directly states the purpose without unnecessary words. It is front-loaded with the action and clearly conveys the tool's function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool, the description is adequate in stating the core function, but it lacks usage guidelines, exclusions, and any detail about the output structure (although an output schema exists). Given the ambiguity with siblings and lack of annotations, the description is not fully complete, but it is not severely deficient for such a simple operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has one parameter 'directory' with no description (0% schema coverage). The description partially compensates by explaining that reports are listed in a specified directory, but it does not clarify the expected format or any constraints on the directory path. This is minimal but non-trivial compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (list) and the resource (available reports) with a scope (specified directory). It is distinct from sibling tool list_available_templates, which focuses on templates, so the purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like list_available_templates or upload_and_generate_prompt. The description provides no exclusions, prerequisites, or context that would help an agent choose this tool over others.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_available_templatesA

List all available prompt templates in the prompts/ directory.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must carry the full burden of behavioral disclosure. It only states the action and directory, but does not explicitly confirm the operation is read-only, mention any limitations (e.g., empty directory behavior), permissions, or side effects. The verb 'List' implies non-mutating behavior, but the description lacks explicit transparency about behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is front-loaded with the action (List) and resource (prompt templates), and adds the directory location. Every word is necessary and there is zero fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with no parameters and an existing output schema, the description is complete. It clearly specifies what is listed and where, and the sibling context (list_available_reports) makes the differentiation obvious without needing extra detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

This tool has zero parameters, so the baseline is 4. There are no parameter semantics to clarify beyond the schema, which is already complete (100% coverage).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists all available prompt templates in the prompts/ directory, using the specific verb 'List' and a precise resource. It naturally distinguishes itself from siblings like list_available_reports (reports vs. templates) and upload_and_generate_prompt (listing vs. generating).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context—listing prompt templates from a specific directory—which implicitly tells the agent when to use this tool over siblings. However, it does not explicitly mention exclusions or alternatives, such as 'use list_available_reports for reports', so it falls short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

upload_and_generate_promptA

Upload an aviation accident report (PDF, DOCX, TXT), parse it, and wrap it in a specified prompt template.

ParametersJSON Schema
NameRequiredDescriptionDefault
file_pathYesPath to the document file.
template_nameNoName of the template in the prompts/ directory (e.g., 'extract_entities').extract_entities

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses the main behavioral steps (upload, parse, wrap) but does not mention side effects, permissions, or whether the file is stored. This is adequate for a simple tool but leaves some uncertainty.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that efficiently conveys the tool's purpose without unnecessary words. It front-loads the main action and includes only essential details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is relatively simple, the schema covers all parameters, and an output schema exists. The description is complete enough for an agent to know how to invoke it, though it could mention that the template must exist (a gap covered by sibling tools).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers both parameters with full descriptions (100% coverage). The description adds useful context about accepted file types, which goes slightly beyond the schema, but it does not significantly enhance understanding of the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (upload), the resource (aviation accident report), accepted formats (PDF, DOCX, TXT), and the processing steps (parse and wrap in a prompt template). It distinguishes itself from sibling tools that only list templates or reports.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool: when you have a report file to upload and need to generate a prompt. Context is clear, though it does not explicitly mention alternatives like using 'list_available_templates' if only browsing templates.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.1.0
    • First observedlist_available_reports
    • First observedlist_available_templates
    • First observedupload_and_generate_prompt
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A3.7/5.0

Scored across 3 tools

Disambiguation5/5

Each tool has a clearly distinct purpose: uploading and generating a prompt, listing templates, and listing reports. There is no overlap or ambiguity between them.

Naming Consistency4/5

All tool names use snake_case and follow a verb-first pattern. The first tool uses a compound verb 'upload_and_generate' while the others use 'list_available', which is a minor deviation but still predictable and readable.

Tool Count4/5

Three tools is a slightly thin but reasonable set for the narrow scope of uploading reports and managing available templates/reports. It is within the acceptable range and does not feel excessive.

Completeness2/5

The tool surface is incomplete for a server named 'aviation-accident-analysis'. It only supports uploading a report and listing templates/reports, with no tools to actually perform analysis, manage templates (create/update/delete), or retrieve detailed content. This leaves significant gaps that would require external actions.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • pymupdf4llm-mcp by pymupdf

    pymupdf4llm-mcpofficial

    pymupdf
    A
    license
    Not graded
    quality
    B
    maintenance
    MCP server for exporting PDF to markdown, optimized for LLM consumption.
    1,982 PyPI
    71
    AGPL 3.0

  • docsearch

    AllenComm
    A
    license
    A
    quality
    D
    maintenance
    MCP server that enables searching and reading binary document files (PDF, DOCX, PPTX, XLSX, ODT, ODS, ODP, RTF, EPUB) using regex patterns and retrieving content by sections.
    2
    MIT

  • Xberg MCP Server

    jamon8888
    A
    license
    Not graded
    quality
    B
    maintenance
    MCP server that extracts clean text, tables, and structured data from documents, images, code, and audio files, supporting 97 formats with OCR, transcription, and code intelligence.
    MIT