Skip to main content
Glama

ClickUp Operator

by noah-vh
parquet.cpython-312.pyc15.7 kB
� lMg�K���ddlZddlZddlZddlmZddlmZ dd�Z dd�Z dd�Z d�Z d �Z d �Z Gd �d �Z Gd �d�Zy)�N�)� url_to_fs)�merge_offset_rangesc �~�|�t|fi|xsi��d}|�!t|�dk(r|j||��St|�}t |g|||||| | | �� } | rt t | ��n|}| jdi�j�}|j|f|di|�| j|i�|d��d�| ��S)a� Return a file-like object for a single Parquet file. The specified parquet `engine` will be used to parse the footer metadata, and determine the required byte ranges from the file. The target path will then be opened with the "parts" (`KnownPartsOfAFile`) caching strategy. Note that this method is intended for usage with remote file systems, and is unlikely to improve parquet-read performance on local file systems. Parameters ---------- path: str Target file path. mode: str, optional Mode option to be passed through to `fs.open`. Default is "rb". metadata: Any, optional Parquet metadata object. Object type must be supported by the backend parquet engine. For now, only the "fastparquet" engine supports an explicit `ParquetFile` metadata object. If a metadata object is supplied, the remote footer metadata will not need to be transferred into local memory. fs: AbstractFileSystem, optional Filesystem object to use for opening the file. If nothing is specified, an `AbstractFileSystem` object will be inferred. engine : str, default "auto" Parquet engine to use for metadata parsing. Allowed options include "fastparquet", "pyarrow", and "auto". The specified engine must be installed in the current environment. If "auto" is specified, and both engines are installed, "fastparquet" will take precedence over "pyarrow". columns: list, optional List of all column names that may be read from the file. row_groups : list, optional List of all row-groups that may be read from the file. This may be a list of row-group indices (integers), or it may be a list of `RowGroup` metadata objects (if the "fastparquet" engine is used). storage_options : dict, optional Used to generate an `AbstractFileSystem` object if `fs` was not specified. strict : bool, optional Whether the resulting `KnownPartsOfAFile` cache should fetch reads that go beyond a known byte-range boundary. If `False` (the default), any read that ends outside a known part will be zero padded. Note that using `strict=True` may be useful for debugging. max_gap : int, optional Neighboring byte ranges will only be merged when their inter-range gap is <= `max_gap`. Default is 64KB. max_block : int, optional Neighboring byte ranges will only be merged when the size of the aggregated range is <= `max_block`. Default is 256MB. footer_sample_size : int, optional Number of bytes to read from the end of the path to look for the footer metadata. If the sampled bytes do not contain the footer, a second read request will be required, and performance will suffer. Default is 1MB. **kwargs : Optional key-word arguments to pass to `fs.open` r)�mode)�metadata�columns� row_groups�engine�max_gap� max_block�footer_sample_size� cache_options�parts)�data�strict)r� cache_typer) r�len�open� _set_engine�_get_parquet_byte_ranges�next�iter�pop�copy�get)�pathr�fsrr r �storage_optionsrr r r r�kwargsr�fn�optionss �ZC:\Users\noahv\Documents\GitHub\clickup-operator\.venv\Lib\site-packages\fsspec/parquet.py�open_parquet_filer$s���b �z� �t� 7�� 5�2� 7�� :����s�7�|�q�0��w�w�t�$�w�'�'��� �F� $� �� �������-� �D�"��d�4�j� �t�B��j�j��"�-�2�2�4�G� �2�7�7� � � �� �� ��H�H�R��$�� � � � � �c ��t|t�r t|�}|�t|||||||��S|j |�} i} g} g} g} d}|�v|�tt |�D]b\}}i| |<t d| ||�D]E}| j|�| j|�| jt||z| |���G�dd}�n�g}g}t |�D]<\}}|j| |�td| ||z �}|j|��>|j|||�}|j�}d}t |�D]I\}}tj||ddd�}| ||dzz }|||ks�6|||<t||dz�}�K|rPtjd |�d ��t |j|||��D]\}}|||z||<||||<�t |�D]�\}}| ||kr?||dkDr6| j|�| jd�| j||��M|j!||||||� �\}}| |gt#|�zz } | |z } | |z } ��t%| | | ||d� �\} } } t |�D]\}}||||f||i| |<�t'|| | | | �|r t)| �| S) aGet a dictionary of the known byte ranges needed to read a specific column/row-group selection from a Parquet dataset. Each value in the output dictionary is intended for use as the `data` argument for the `KnownPartsOfAFile` caching strategy of a single path. )r r r r TrFi����������little�zYNot enough data was used to sample the parquet footer. Try setting footer_sample_size >= �.)r �footer� footer_start�r r �sort)� isinstance�strr�&_get_parquet_byte_ranges_from_metadata�sizes� enumerate�range�append�min�max� cat_rangesr�int� from_bytes�warnings�warn�_parquet_byte_rangesrr�_transfer_ranges�_add_header_magic)�pathsrrr r r r rr � file_sizes�result� data_paths� data_starts� data_ends�add_header_magic�ir�b� footer_starts� footer_ends� sample_size�footer_samples�missing_footer_starts� large_footer� footer_size�real_footer_start�block�path_data_starts�path_data_endss r#rr�s���&�&�#���V�$����6� � � ��!��� � ����%��J��F��J��K��I�����:�-� !��'�G�A�t��F�4�L��1�j��m�Y�7���!�!�$�'��"�"�1�%�� � ��Q��]�J�q�M�!B�C�8�(� !��� �� � ��'�G�A�t� � � �z�!�}� -��a��A��1C�!C�D�K� � � �� -�(����u�m�[�I��!.� 2� 2� 4��� � ��'�G�A�t��.�.���):�2�b�)A�8�L�K� *�1� ��q�� A� � �=��#3�3�+<�%�a�(�"�<�+��/�C� � (� � �M�M�5�5A�N�!�E� �&�� � ��)�!�����5�%*�N�1�,=�$=��q�!�#8��#;� �a� ��!��'�G�A�t��!�}�y�(� ��#�a�'��%�%�d�+��&�&�q�)��$�$�]�1�%5�6��06�/J�/J��%�%�a�(�*�1�-� 0K�0� ,� �n� �4�&�3�'7�#8�8� 8�J� �+� +�K� �� '�I�/(�4.A� � � ���� . �*� �K��!��'�G�A�t�*�1�-�{�1�~�>��q�@Q�R�F�4�L�(��R���[�)�D���&�!� �Mr%c���|j|||��\}}} t||| ||d��\}}} tt|��D� cic]} | i��} } t || ||| �t | �| Scc} w)z�Simplified version of `_get_parquet_byte_ranges` for the case that an engine-specific `metadata` object is provided, and the remote footer metadata does not need to be transferred before calculating the required byte ranges. )r rFr-)r=r�list�setr>r?) rrr r r r r rCrDrEr!rBs r#r1r1s��� *0�)D�)D����*E�*�&�J� �Y�*=������ � *�&�J� �Y� $�C� �O�4� 5�4��b�"�f�4�F� 5��R���[�)�D��f�� �M�� 6s� A,c�j�|||f}tg|�|j|����D]\}}}} | ||||f<�y�N)�zipr8) r�blocksr@�starts�ends�rangesr�start�stoprs r#r>r>HsL���V�T� "�F�#&�#G��#G� �� � �v�0F�#G���e�T�4�&*��t� �e�T�]�#�$Hr%c��t|j��D]<}d}||j�D]}|ddk(s� |ddk\s�d}n|s�5d||d<�>y)NTrr�FsPAR1)rra)rU�keys)rr� add_magic�ks r#r?r?Osa���T�Y�Y�[�!��� ��d����"�A���t�q�y�Q�q�T�Q�Y�!� ��#� �!(�D��J�v� �"r%c��|dk(rd}n0t|t�s td��|dvrt|�d���|g}|D]%} |dk(r t�cS|dk(r t �cS�'t d|�d���#t $rY�BwxYw) N�auto)� fastparquet�pyarrowzMFailed to set parquet engine! Please pass 'fastparquet', 'pyarrow', or 'auto'z) engine not supported by `fsspec.parquet`rgrhzLThe following parquet engines are not installed in your python environment: zS.Please install 'fastparquert' or 'pyarrow' to utilize the `fsspec.parquet` module.)r/r0� ValueError�FastparquetEngine� PyarrowEngine� ImportError)� engine_str� try_enginesr s r#rr[s����V��0� � � �C� (�� >� � � �5� 5��J�<�'P�Q�R�R�!�l� ��� ���&�(�*�*��9�$�$��&�%� � � '�'2�m�4/� 0� ��� � � � �s�A2�A2�2 A>�=A>c�(�eZdZd�Zd�Z dd�Zy)rjc��ddl}||_y�Nr)rg�fp)�selfrrs r#�__init__zFastparquetEngine.__init__�s �� ���r%c�$�|j|�SrX)�row_group_filename)rs� row_group�pfs r#�_row_group_filenamez%FastparquetEngine._row_group_filename�s���$�$�Y�/�/r%Nc ��|}ggg} }}|�.|jjtj|��}|�dn t |�} | �St |d�rG|j jdg�D� cgc]} t| t�s| ��} } | t | �z} |rt|dt�sd} n|} |j}t|�D]�\}}| �|| vs� |j||�}|jD]�}|jj d}| �|| vs�#|jj"}|�|jj$}|jj&}|�||ks�o|j)|�|j)|�| j)t+||z|xs||z������|r||| fS|| fScc} w)N�pandas_metadata� index_columnsr)rr� ParquetFile�io�BytesIOrV�hasattrr{rr/�dictr9r r3ryr � meta_data�path_in_schema�dictionary_page_offset�data_page_offset�total_compressed_sizer5r6)rsr r rr+r,rxrCrDrE� column_set�ind�md_index�row_group_indices�rrwr!�column�name� file_offset0� num_bytess r#r=z&FastparquetEngine._parquet_byte_ranges�s�����-/��R��K� � �:����$�$�R�Z�Z��%7�8�B�%�_�T�#�g�,� � � !�g�b�2C�&D��-�-�1�1�/�2�F��F�C�!�#�t�,��F� �� �#�h�-� '�J� �j��A���<� $� �!+� ����J�&�j�1�L�A�y�!�(�A�1B�,B��-�-�i��<��'�/�/�F�!�+�+�:�:�1�=�D�"�)�T�Z�-?�'-�'7�'7�'N�'N� �'�/�+1�+;�+;�+L�+L�L�$*�$4�$4�$J�$J� �'�/�<�,�3N�&�-�-�b�1�'�.�.�|�<�%�,�,� #�$0�9�$<�$0�$N�\�I�5M�!"��0�2�4 ��{�I�5� 5��I�%�%��cs�4F>�NNNN��__name__� __module__� __qualname__rtryr=�r%r#rjrj~s ��� 0� ���� C&r%rjc�(�eZdZd�Zd�Z dd�Zy)rkc��ddlm}||_yrq)�pyarrow.parquet�parquet�pq)rsr�s r#rtzPyarrowEngine.__init__�s ��$���r%c��t�rX)�NotImplementedError)rsrwrs r#ryz!PyarrowEngine._row_group_filename�s��!�!r%Nc���|� td��gg}}|jjtj|��j }|�dn t |�} | ��|jj�} | j duxrd| j v} | rltj| j djd��jdg�D� cgc]} t| t�s| ��} } | t | �z} t|j �D]�}|�||vs� |j#|�}t|j$�D]�}|j'|�}|j(}|j+d�d}| � || vs|| vs�?|j,}|� |j.}|j0}||ks�k|j3|�|j3t5||z|������||fScc} w)Nz.metadata input not supported for PyarrowEnginespandas�utf8r|r*r)rir�r}r~rrrV�schema�to_arrow_schema�json�loads�decoderr/r�r4�num_row_groupsrw� num_columnsr�r��splitr�r�r�r5r6)rsr r rr+r,rDrE�mdr�r��has_pandas_metadatar�r�r�rw�cr�r�� split_namer�r�s r#r=z"PyarrowEngine._parquet_byte_ranges�s��� � ��M�N� N�!#�R�Y� � �W�W� � ����F�!3� 4� =� =��%�_�T�#�g�,� � � !��Y�Y�.�.�0�F����t�+�L� �V�_�_�0L� �#� $�z�z���� �2�9�9�&�A� ��c�/�2�.� /�� /��&�c�4�0� � /����c�(�m�+� ��r�(�(�)�A��!�Q�*�_��L�L��O� ��y�4�4�5�A�&�-�-�a�0�F�!�0�0�D�"&���C���!3�J�"�*��:�-�%��3�'-�'D�'D� �'�/�+1�+B�+B�L�$*�$@�$@� �'�,�6�'�.�.�|�<�%�,�,� #�L�9�$<�l� K��#6� *�2�I�%�%��Is�G(r�r�r�r%r#rkrk�s��� "� ���� ;&r%rk) �rbNNNNNFrf���@B�@B)NNNr�r�r�rf)NNr�r�)r~r�r;�corer�utilsrr$rr1r>r?rrjrkr�r%r#�<module>r�s��� � ���&� � � � ��� � � �� �y�~� �� �� � �N�j �� ��'�T+� )� �FR&�R&�jJ&�J&r%

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/noah-vh/mcp-server-clickup'

If you have feedback or need assistance with the MCP directory API, please join our Discord server