llama_index_manager.cpython-310.pyc•8.37 kB
o
�|�g�* � @ s� d Z ddlZddlZddlZddlmZmZmZmZm Z m
Z
m Z m Z ddl
Z
ddlmZmZmZ ddlmZ ddlmZ ddlmZ ddlmZ ejejd � e� e �Z G d
d � d �Z dS ) z�
LlamaIndex Manager Module
Manages the LlamaIndex integration for the persistent-code knowledge graph.
Provides utilities for semantic search, embedding generation, and knowledge graph operations.
� N)�Dict�List�Optional�Any�Tuple�Set�Union�Callable)�Document�KnowledgeGraphIndex�StorageContext)�SimpleDocumentStore)�SimpleIndexStore)�
BaseEmbedding)�HuggingFaceEmbedding)�levelc @ s� e Zd ZdZ d"dededefdd�Zdefd d
�Zdefd d �Z d
e
defdd�Z d
e
defdd�Z dedededefdd�Z
d#dededeeeeeef f fdd�Zdefd d �Zdefd d �Zdeeef fd d!�ZdS )$�LlamaIndexManagerz#Manager for LlamaIndex integration.N� project_name� project_dir�triple_extract_fnc C s0 || _ || _|| _|| _d| _d| _| �� dS )a% Initialize the LlamaIndex manager.
Args:
project_name: Name of the project
project_dir: Directory for project storage
config_instance: Configuration instance
triple_extract_fn: Function to extract triples from documents
N)r r �configr �kg_index� embed_model�_initialize_llama_index)�selfr r �config_instancer � r �^/Users/sparshdrolia/Sparsh personal/persistent-code-mcp/persistent_code/llama_index_manager.py�__init__ s
zLlamaIndexManager.__init__�returnc
C s� | j �� s t�d� dS z4| j �� }t�d|� �� t|d�| _tjt � t
� d�}t g || j| j dd�| _
t�d| j� �� W dS tyc } zt�d t|�� �� d
| _
d
| _W Y d
}~dS d
}~ww ) zInitialize LlamaIndex components.
Returns:
success: Whether initialization was successful
�3LlamaIndex integration is disabled in configurationF� Initializing embedding model: ��
model_name)�docstoreZ index_storeT��storage_contextr �kg_triple_extract_fn�include_embeddingsz3LlamaIndex Knowledge Graph initialized for project z!Failed to initialize LlamaIndex: N)r �is_llama_index_enabled�logger�info�get_embedding_modelr r r �
from_defaultsr
r r r r r � Exception�warning�str)r r# r& �er r r r 5 s6
����z)LlamaIndexManager._initialize_llama_indexc C s | j duo | jduS )z�Check if LlamaIndex is available and initialized.
Returns:
is_available: Whether LlamaIndex is available
N)r r )r r r r � is_available] s z LlamaIndexManager.is_available�documentc
C �n | � � sdS z| j�|� t�d|j� d�� W dS ty6 } zt�dt|�� �� W Y d}~dS d}~ww )z�Add a document to the knowledge graph.
Args:
document: LlamaIndex document
Returns:
success: Whether the document was added successfully
FzAdded document z to LlamaIndex KGTz&Failed to add document to LlamaIndex: N) r2 r �insertr* r+ �doc_idr. r/ r0 �r r3 r1 r r r � add_documente � ��z LlamaIndexManager.add_documentc
C r4 )z�Update a document in the knowledge graph.
Args:
document: LlamaIndex document
Returns:
success: Whether the document was updated successfully
FzUpdated document z in LlamaIndex KGTz)Failed to update document in LlamaIndex: N) r2 r �updater* r+ r6 r. r/ r0 r7 r r r �update_documentz r9 z!LlamaIndexManager.update_document�subject� predicate� object_textc
C sz | � � sdS z| j�|||� t�d|� d|� d|� �� W dS ty< } zt�dt|�� �� W Y d}~dS d}~ww )a Add a knowledge triple to the graph.
Args:
subject: Subject entity
predicate: Relationship predicate
object_text: Object entity
Returns:
success: Whether the triple was added successfully
FzAdded triple: � Tz$Failed to add triple to LlamaIndex: N)r2 r Z upsert_triplet_and_embeddingr* r+ r. r/ r0 )r r< r= r> r1 r r r �
add_triple� s � ��z LlamaIndexManager.add_triple� �query�similarity_top_kc
C s� | � � s t�d� g S z@| j�|�}| jj|d�}|�|�}g }|D ]}|jj }|r:|�
|j ||jj |jj
d�f� q"t�d|� dt|�� d�� |W S tyi } zt�dt| �� �� g W Y d} ~ S d} ~ ww ) z�Perform semantic search using LlamaIndex.
Args:
query: Search query
similarity_top_k: Number of results to return
Returns:
results: List of (score, node) tuples
z8Semantic search unavailable - LlamaIndex not initialized)rC )�id�text�metadatazSemantic search for 'z' found z resultszSemantic search failed: N)r2 r* r/ r Zget_text_embeddingr Z as_retrieverZretrieve�nodeZ
ref_doc_id�append�scorerE rF r+ �lenr. r0 )
r rB rC Zquery_embeddingZ retriever�resultsZprocessed_results�resultr6 r1 r r r �semantic_search� s4
�
�� ��z!LlamaIndexManager.semantic_searchc
C s� | � � sdS z"tj�| jd�}tj|dd� | jjj|d� t �
d|� �� W dS t yE } zt � dt
|�� �� W Y d}~dS d}~ww ) z�Save the LlamaIndex knowledge graph to disk.
Returns:
success: Whether the save was successful
F� llama_indexT)�exist_ok�� persist_dirzSaved LlamaIndex to zFailed to save LlamaIndex: N)r2 �os�path�joinr �makedirsr r& �persistr* r+ r. r/ r0 )r rQ r1 r r r �save� s ��zLlamaIndexManager.savec
C s� | j �� s t�d� dS tj�| jd�}tj�|�s$t�d|� �� dS z5| j du r=| j �
� }t�d|� �� t |d�| _ t j
|d�}tj|| j | jd d
�| _t�d |� �� W d S tyy } zt�d t|�� �� d| _W Y d}~dS d}~ww )
z�Load the LlamaIndex knowledge graph from disk.
Returns:
success: Whether the load was successful
r FrN z No saved LlamaIndex found at Nr! r" rP Tr% zLoaded LlamaIndex from zFailed to load LlamaIndex: )r r) r* r+ rR rS rT r �existsr r, r r r- r � from_storager r r. r/ r0 )r rQ r# r& r1 r r r �load� s:
����zLlamaIndexManager.loadc
C s� | j �� | �� | j �� r| j �� ndd�}| �� rX| jdurXz | jjj}t|d�r.t|j �nd}|�
|dd�� W |S t yW } z|�
dt |�d�� W Y d}~|S d}~ww |S )zwGet the status of the LlamaIndex integration.
Returns:
status: Status information
N)�enabledZ available�embedding_model�docsr T)Zdocument_count�index_initialized)r^ �error)
r r) r2 r, r r& r$ �hasattrrJ r] r: r. r0 )r �statusr$ Z doc_countr1 r r r �
get_status s, �
�
����z LlamaIndexManager.get_status)N)rA )�__name__�
__module__� __qualname__�__doc__r0 r r �boolr r2 r
r8 r; r@ �intr r �floatr r rM rW rZ rb r r r r r s4 ���
�(���
�.+r ) rf rR �logging�tempfile�typingr r r r r r r r �json�llama_index.corer
r r Z!llama_index.core.storage.docstorer
Z$llama_index.core.storage.index_storer Zllama_index.core.embeddingsr Z"llama_index.embeddings.huggingfacer � basicConfig�INFO� getLoggerrc r* r r r r r �<module> s (