Skip to main content
Glama

MCP VectorStore Server

vectorstore.cpython-311.pyc22 kB
� �@rh�:���ddlZddlZddlZddlZddlmZddlmZddlZddlm Z ddl m Z m Z ddl Z ddlZddlmZejde���ejde���d Zd ejd <e��Ze d ��d edefd���Ze d��d edefd���Ze d��d edefd���Ze d��dedefd���Zd(d�Zd�ZGd�de ��Ze��Zddl m!Z!ddl"m#Z#ddl$m%Z%ddl&m'Z'dd l(m)Z)dd!l*m+Z+ddlZdd"l,m,Z,dd#l-m.Z.ejdd$e�%��Gd&�d'e ��Z/dS))�N)� BeautifulSoup)�datetime)�LLMSherpaFileLoader)�tool�BaseTool)�DuckDuckGoSearchResults�ignore)�categoryzollama/openhermeszeMozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36� USER_AGENTzgoogle search tool.�argument�returnc� �t|��S)zVUse this tool to search new information using google, if you do need more information.)�get_google_content�r s �$/home/em/McpDocServer/vectorstore.py�google_search_toolrs�� �h� '� '�'�zduckduck go search toolc�Z�td|��t�|��}|S)z�Use this tool to search new information by crawling the web, if you do need additional information but do not use json as input!zDuck: )�print�duckduckgo_tool�invoke)r �responses r�duck_duck_go_search_toolrs,�� �(�H�����%�%�h�/�/�H� �Orzwikipedia search toolc� �t|��S)zcUse this tool to search a single keyword on wikipedia, if you do not know the answer to a question.)�get_wiki_contentrs r�wikipedia_search_toolr%s�� �H� %� %�%r� calculator� operationc� �t|��S)z>Use this tool to calculate result of a mathematical expression)�eval)rs r�calculate_toolr!*s�� � �?�?�r�c���tjdtj���d���d������}dd|��d�}d|z}t j||���}t|j d ��}|� d � ��D]%}t|j ��td ���&dd � |� ����z}td|d��tt j||���j d��}|�d��d|�D�]}t|j�d�������|�d���d��} | �d��r5| � dd��d� d��d} t| ��|�d��} | dkr4t| �d�������td����dS)Nz gws_z%Y%m%du-0_RC3en €º¦±z�Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 Edg/123.0.2420.97zSOCS=)z User-Agent�Cookiezhttps://google.com/search?q=)�headers�lxml�g)�class_z-----�+�From� --- z html.parserzdiv:has(>div>a[href] h3)� za[href]:has(h3)�hrefz/url?q=�=��&rzdiv:has(>a[href] h3)+div)�base64� encodebytesr�today�strftime�encode�requests�getr�text�find_allr�join�split�select�h3�get_text�strip� select_one� startswith) r8� max_results� socs_cookier%�urlr�soupr'�d�res_link�sel_typs rrr0sT���$�&[���HX�HX�Ha�Ha�bj�Hk�Hk�&[�&[�&[�&b�&b�&d�&d�e�e�K� L�'�+�'�'���G� )�4� /�C��|�C��1�1�1�H� ���� /� /�D� �]�]�#�]� &� &���� �a�f� � � � �g����� )�3�8�8�D�J�J�L�L�+A�+A� A�C� �&�#�y�!�!�!� ���c�7�;�;�;�@�-� P� P�D��[�[�3� 4� 4�\�k�\� B���� �a�d�m�m�C� � �&�&�(�(�)�)�)��<�<� 1�2�2�6�6�v�>�>�� � � �y� )� )� >��~�~�c�!�,�,�Q�/�5�5�c�:�:�1�=�H� �h�����,�,�9�:�:�� �d�?�?� �'�"�"�3�'�'�-�-�/�/� 0� 0� 0� �i������rc�`� tjdd��}|�|��}|���r1t d|j��t d|j��|jSt d��|jS#t$r}t d|��Yd}~nd}~wwxYwdS)Nz"MyProjectName (merlin@example.com)�enz Page - Title:zPage - Summary:zPage not found.zAn exception occurred:�)� wikipediaapi� Wikipedia�page�existsr�title�summary� Exception)�key_word� wiki_wikirN�errors rrrVs��� /� �*�+O�QU�V�V� ��~�~�h�'�'�� �;�;�=�=� %� �/�4�:� .� .� .� �#�T�\� 2� 2� 2��<� � �#� $� $� $��|��� �/�/�/� �&��.�.�.�.�.�.�.�.�����/���� �2s�A.B�1B� B+�B&�&B+c�8�eZdZUdZeed<dZeed<defd�ZdS)�ReadFile�namezzOnly use this tool if you want to read a file content on this computer. Only execute this tool if you want to read a file!� description�filenamec�(� t|��dkr�|dkrt|��td�� tjd|gddd���}td|����t|j��|jS#tj$r-}dt |����}t|��|cYd}~Sd}~wt$r-}d t |����}t|��|cYd}~Sd}~wwxYwn#t$rYd SwxYwd tfd �}dS) NrrKzFilename found�catT)�capture_outputr8�checkzReading file: zError reading file with cat: zError: zSThis is not a valid python code search syntax. Try a different string based syntax.�radiusc� �td���)N� This tool does not support async��NotImplementedError)�selfr_s r�_arunzReadFile._run.<locals>._arun�s��%�&H�I�I� Ir) �lenr� subprocess�run�stdout�CalledProcessError�strrR�int)rdrZ�result�e� error_msgrUres r�_runz ReadFile._runks��� i��8�}�}��!�!��r�>�>��(�O�O�O��&�'�'�'� %�'�^�U�H�,=�t�[_�hl�m�m�m�F��5�8�5�5�6�6�6��&�-�(�(�(�!�=�(��!�4�%�%�%� H��A��� H� H�I��)�$�$�$�$�$�$�$�$�$�$����� �%�%�%� 6�#�e�*�*� 6� 6�I��)�$�$�$�$�$�$�$�$�$�$�����%����"��$� i� i� i�h�h�h� i���� J�� J� J� J� J� J� JsS�7C6�AB�C2�"B8�2C2�3C6�8 C2�"C-�'C2�(C6�-C2�2C6�6 D�DN)�__name__� __module__� __qualname__rXrk�__annotations__rYrp�rrrWrWgsd��������D�#����T�K��T�T�T�J�S�J�J�J�J�J�JrrW)�LayoutPDFReader)�RecursiveCharacterTextSplitter)�HuggingFaceEmbeddings)�FAISS��Document)�Any)�glob)�Fieldz�Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.11/migration/)�messager c���eZdZUdZeed<dZeed<ed���Zeed<ed���Z e ed <ed���Z e ed <ed���Z e ed <ed���Z e ed <ed���Zeed <�fd�Zd�Zd�Zd�Zd�Zd�Zd�Zd�Zd�Zd�Zd�Zd�Zdefd�Zd&dededefd �Zd!edefd"�Zd#edefd$�Zd#edefd%�Z�xZS)'�PDFVectorStoreTool�pdf_vector_storerXzwUse this tool to read AI ECG classification related PDFs to gain a deep ecg classificaiton knowledge and understanding.rYz8http://localhost:5001/api/parseDocument?renderFormat=all)�default�llmsherpa_api_urlN� pdf_reader� text_splitter� embeddings� vector_store�dirc ���t��jdi|��t|j��|_t ddt gd�dd���|_tj d��� tj ��tdd d id di� ��|_ d|_d |_|�|j��dS)Ni���)z z � TF)� chunk_size� chunk_overlap�length_function� separators�keep_separator�is_separator_regex�sentence_transformersz'sentence-transformers/all-mpnet-base-v2�device�cuda�normalize_embeddings)� model_name� model_kwargs� encode_kwargsz./books/unit_testru)�super�__init__rvr�r�rwrfr��logging� getLogger�setLevel�ERRORrxr�r�r��create_vector_store)rd�data� __class__s �rr�zPDFVectorStoreTool.__init__�s��������� � �4� � � �)�$�*@�A�A���;���� � � �� � � � ���" ��1�2�2�;�;�G�M�J�J�J�0�@�#�F�+�1�4�8�  � � ���!���&��� � � ���*�*�*�*�*rc�$�|jjjS�N�r��index�ntotal�rds r�vectorstore_get_num_itemsz,PDFVectorStoreTool.vectorstore_get_num_items����� �&�-�-rc�r�|jjj���D]\}}||jvr|cS�dS�Nr�r��docstore�_dict�items� page_content�rd�context�_id�docs r�vectorstore_search_id_by_contexz2PDFVectorStoreTool.vectorstore_search_id_by_contex��K���)�2�8�>�>�@�@� � �H�C���#�*�*�*�� � � �+��qrc�n�|jjj���D]\}}||jvrg}�dSr�r��rdr�r�r��id_ids r�vectorstore_delete_by_contexz/PDFVectorStoreTool.vectorstore_delete_by_contex�sH���)�2�8�>�>�@�@� � �H�C���#�*�*�*���� � rc�$�|jjjSr�r�r�s rr�z,PDFVectorStoreTool.vectorstore_get_num_items�r�rc�r�|jjj���D]\}}||jvr|cS�dSr�r�r�s rr�z2PDFVectorStoreTool.vectorstore_search_id_by_contex�r�rc���|jjj���D]G\}}||jvr9g}|�|��|j�|dd����HdS�Nrr/)r�r�r�r�r��append�deleter�s rr�z/PDFVectorStoreTool.vectorstore_delete_by_contex�st���)�2�8�>�>�@�@� 5� 5�H�C���#�*�*�*���� � �S�!�!�!��!�(�(��q��s��4�4�4���qrc��t|jjj��D]f}|jjj���}|D]>\}}g}|�|��|j�|dd����gdSr�) �ranger�r�r�r�r�r�r�r�)rd�k� copy_safer�r�r�s r�vectorstore_clearz$PDFVectorStoreTool.vectorstore_clear�s����t�(�.�5�6�6� � �A��)�2�8�>�>�@�@�I�%� � ���S���� � �S�!�!�!��!�(�(��q��s��4�4�4����qrc�x�g}|�|��|j�|dd���dSr�)r�r�r�)rd�idr�s r�vectorstore_delete_by_idz+PDFVectorStoreTool.vectorstore_delete_by_id�s>���� � � �R���� �� � ��q��s��,�,�,�,�,rc��d}|jjj���D]�\}}t |||��t |jj�����t |jj�����t |jj�d����|dz }��dSr�)r�r�r�r�r�index_to_docstore_id�valuesr7)rdr�r�r�s r�vectorstore_dumpz#PDFVectorStoreTool.vectorstore_dump�s��� ���)�2�8�>�>�@�@� � �H�C�� �!�S�#� � � � �$�#�8�>�>�@�@� A� A� A� �$�#�8�?�?�A�A� B� B� B� �$�#�8�<�<�Q�?�?� @� @� @� ��F�A�A�  � rc�J�|j�|jdz��dS)N�/vectorstore/vectorstore.db)r�� save_localr�r�s r�save_vector_storez$PDFVectorStoreTool.save_vector_store�s'�� ��$�$�T�X�0M�%M�N�N�N�N�Nrc�� tj|jdz|jd���|_dS#t $r}t d|��Yd}~dSd}~wwxYw)Nr�T)�allow_dangerous_deserializationz"Error: Loading vectorstore failed:)ry� load_localr�r�r�rRr)rdrns r�load_vector_storez$PDFVectorStoreTool.load_vector_store�s��� ;� %� 0���<Y�1Y�[_�[j�MQ�!R�!R�!R�D� � � ��� ;� ;� ;� �6�� :� :� :� :� :� :� :� :� :����� ;���s�)-� A�A � A� input_queryc ��|j�|d���}|D],\}}td|d�d|j�d|j�d�dz���-dS) Nr/)�queryr�z* [SIM=�3fz] z [�]r+)r��similarity_search_with_scorerr��metadata)rdr��resultsr��scores r�search_vectorstorez%PDFVectorStoreTool.search_vectorstoresy���#�@�@�{�UV�@�W�W��!� Y� Y�J�C�� �K�E�K�K�K��(8�K�K�C�L�K�K�K�i�W� X� X� X� X� Y� Yr�r�r�r c�v� |jsdS|j�||���}g}|D]@}|j�dd��}|j}|�d|�d|�d����Ad�|��}t|��|S#t$r} d t| ����cYd } ~ Sd } ~ wwxYw) z,Search the vector store for relevant contentz0No vector store exists. Please create one first.)r��sourcezunknown sourcezFrom z: r�r+zError searching vector store: N) r��similarity_searchr�r7r�r�r:rrRrk) rdr�r�r��formatted_resultsr�r��content�retrns r�vectorstore_searchz%PDFVectorStoreTool.vectorstore_search s��� =��$� J�I�I��'�9�9�%�1�9�E�E�G� "� �� I� I����)�)�(�4D�E�E���*��!�(�(�)G��)G�)G�G�)G�)G�)G�H�H�H�H��.�.�!2�3�3�C� �#�J�J�J��J��� =� =� =�<�C��F�F�<�<� <� <� <� <� <� <����� =���s#�B�BB� B8�B3�-B8�3B8�directory_pathc ��� � tj�|jdz���s�tj�|��sd|��St tj�|d����}|sd|��St dt|���d|����g}|D]�� t d� ����|j� � ��}g}|� ��D])}|� |� �����*d�|��}|j �|��}� fd�|D��} |�| ����#t $r,} t d � �d t#| ������Yd } ~ ��d } ~ wwxYw|sd St%j||j��|_t d ��tj|jdzd���|���n|���t d��d S#t $r*} t dt#| ������Yd } ~ d Sd } ~ wwxYw)z8Create a FAISS vector store from all PDFs in a directoryz /vectorstorezError: Directory not found at z*.pdfzNo PDF files found in z Processing z PDF files from r�c�6��g|]}t|d�i�����S)r�)r�r�rz)�.0�t�pdf_paths �r� <listcomp>z:PDFVectorStoreTool.create_vector_store.<locals>.<listcomp>>s-���$l�$l�$l�ab�X�1��RZ�G[�%\�%\�%\�$l�$l�$lrzError processing z: Nz(No documents were successfully processedzDirectory not foundT)�exist_okz6Vector store created successfully from PDFs and saved.zError creating vector store: )�os�pathrOr�r}r:rrfr��read_pdf�chunksr��to_context_textr�� split_text�extendrRrkry�from_documentsr�r��makedirsr�r�) rdr�� pdf_files� all_documentsr�� text_content�chunk� full_content�texts� documentsrnr�s @rr�z&PDFVectorStoreTool.create_vector_stores����9 <��7�>�>�$�(�^�";�<�<�+ )��w�~�~�n�5�5�M�L�N�L�L�L�!�����n�g�!F�!F�G�G� � �E�D�N�D�D�D��T�C� �N�N�T�T�N�T�T�U�U�U� "� � )�!�!�H�!��6�H�6�6�7�7�7�"�o�6�6�x�@�@��(*� �%(�Z�Z�\�\�I�I�E�(�/�/��0E�0E�0G�0G�H�H�H�H�(,�y�y��'>�'>� � $� 2� =� =�l� K� K��$l�$l�$l�$l�fk�$l�$l�$l� �%�,�,�Y�7�7�7�7��%�!�!�!��F�(�F�F�c�!�f�f�F�F�G�G�G� ���������!����%�F�E�E�$)�$8����$X�$X��!��+�,�,�,�� �D�H�~�5��E�E�E�E��&�&�(�(�(�(��&�&�(�(�(� �K� L� L� L� L� L��� <� <� <� �:�#�a�&�&�:�:� ;� ;� ;� ;� ;� ;� ;� ;� ;����� <���sV�A H.�3H.�(H.�,B>E+�*H.�+ F!�5"F�H.�F!�!H.�(BH.�. I"�8I�I"�commandc�� |�|��S#t$r}dt|����cYd}~Sd}~wwxYw)zRun the tool with query commandzError processing command: N)r�rRrk)rdrrns rrpzPDFVectorStoreTool._run]s]�� 9��*�*�7�3�3� 3��� 9� 9� 9�8��A���8�8� 8� 8� 8� 8� 8� 8����� 9���s�� =�8�=�=c� �td���)zAsync implementationrarb)rdrs rrezPDFVectorStoreTool._arunds��!�"D�E�E�Er)r�)rqrrrsrXrkrtrYr~r�r�r|r�r�r�r�r�r�r�r�r�r�r�r�r�r�rlr�r�rpre� __classcell__)r�s@rr�r��s���������"�D�#�"�"�"�Q�K��Q�Q�Q�#�U�+e�f�f�f��s�f�f�f��e�D�)�)�)�J��)�)�)���t�,�,�,�M�3�,�,�,��e�D�)�)�)�J��)�)�)���d�+�+�+�L�#�+�+�+��u�T�"�"�"�C��"�"�"�"+�"+�"+�"+�"+�H.�.�.���� ��� .�.�.���� ������-�-�-� ���O�O�O�;�;�;�Y�c�Y�Y�Y�Y� =�=��=��=�C�=�=�=�=�,;<�#�;<�#�;<�;<�;<�;<�|9�C�9�C�9�9�9�9�F�S�F�S�F�F�F�F�F�F�F�Frr�)r")0rLrgr�r6�bs4rrr1�.langchain_community.document_loaders.llmsherpar� crewai_toolsrrr��warnings�langchain_community.toolsr�filterwarnings�DeprecationWarning� FutureWarning� ollama_model�environrrkrrrr!rrrW�read_file_tool�llmsherpa.readersrv�langchain.text_splitterrw�langchain.embeddingsrx�langchain.vectorstoresry�langchain.docstore.documentr{�typingr|r}�pydanticr~r�rurr�<module>rs[���������� � � � ����������������� � � � �N�N�N�N�N�N�'�'�'�'�'�'�'�'���������=�=�=�=�=�=�����+=�>�>�>�>�����=�9�9�9�9�"� �C�� �<��)�)�+�+�������(��(��(�(�(���(� �� �!�!��s��s����"�!�� �����&�C�&�C�&�&�&���&���l����c��c������� $�$�$�$�L���"J�J�J�J�J�x�J�J�J�:�����.�-�-�-�-�-�B�B�B�B�B�B�6�6�6�6�6�6�(�(�(�(�(�(�0�0�0�0�0�0������� � � � �����������������+v�AS�T�T�T�T�TF�TF�TF�TF�TF��TF�TF�TF�TF�TFr

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/BierschneiderEmanuel/McpDocServer'

If you have feedback or need assistance with the MCP directory API, please join our Discord server