�
lMgr � � � d dl Z d dlmZmZ d dlmZ d dlmZ d dlm Z m
Z
d dl m Z d� Z
d� Z G d � d
e
� Z G d � d e � Zy)
� N)�Client�_get_global_client)�Worker)�
filesystem)�AbstractBufferedFile�AbstractFileSystem��infer_storage_optionsc �T � | �
t � S t | t � r| S t | � S �N)r �
isinstancer )�clients �gC:\Users\noahv\Documents\GitHub\clickup-operator\.venv\Lib\site-packages\fsspec/implementations/dask.py� _get_clientr
s+ � �
�~�!�#�#� �F�F� #��
� �f�~� � c �4 � t t j � S r )�boolr �
_instances� r r �
_in_workerr s � � ��!�!� "�"r c �p � � e Zd ZdZ d � fd� Zed� � Zd� Zd� Zd� Z d� Z
d� Z d � Z d
d
�Z
d � Z� xZS )�DaskWorkerFileSystema) View files accessible to a worker as any other remote file-system
When instances are run on the worker, uses the real filesystem. When
run on the client, they call the worker to provide information or data.
**Warning** this implementation is experimental, and read-only for now.
c � �� t �| � di |�� |d u |d u z s t d� �|| _ || _ d | _ || _ || _ | j � y )NzKPlease provide one of filesystem instance (fs) or target_protocol, not bothr ) �super�__init__�
ValueError�target_protocol�target_options�workerr �fs�_determine_worker)�selfr r r r �kwargs� __class__s �r r z DaskWorkerFileSystem.__init__! sk �� � ���"�6�"��d�
��$� 6�7� �-��
� /�� � ,����� � �� ���� � � � r c �J � t | � }d|v rd|v rd|d � d|d � �iS i S )N�host�portr �:r )�path�sos r �_get_kwargs_from_urlsz*DaskWorkerFileSystem._get_kwargs_from_urls1 s< � �
"�4�
(�� �R�<�F�b�L� ��F�� �A�b��j�\� :�;� ;��Ir c �
� t � r>d| _ | j �*t | j fi | j
xs i ��| _ y y d| _ t
| j � | _ t j | � | _
y )NTF) r r r r r r r r �dask�delayed�rfs�r" s r r! z&DaskWorkerFileSystem._determine_worker9 sm � � �<� �D�K��w�w��$��(�(��-1�-@�-@�-F�B���� �
�D�K�%�d�k�k�2�D�K��|�|�D�)�D�Hr c � � | j r | j j |i |�� y | j j |i |��j � y r )r r �mkdirr/ �compute�r" �argsr# s r r2 zDaskWorkerFileSystem.mkdirE s@ � � �;�;� �D�G�G�M�M�4� *�6� *� �D�H�H�N�N�D� +�F� +� 3� 3� 5r c � � | j r | j j |i |�� y | j j |i |��j � y r )r r �rmr/ r3 r4 s r r7 zDaskWorkerFileSystem.rmK �@ � � �;�;� �D�G�G�J�J�� '�� '� �D�H�H�K�K�� (�� (� 0� 0� 2r c � � | j r | j j |i |�� y | j j |i |��j � y r )r r �copyr/ r3 r4 s r r: zDaskWorkerFileSystem.copyQ s@ � � �;�;� �D�G�G�L�L�$� )�&� )� �D�H�H�M�M�4� *�6� *� 2� 2� 4r c � � | j r | j j |i |�� y | j j |i |��j � y r )r r �mvr/ r3 r4 s r r< zDaskWorkerFileSystem.mvW r8 r c � � | j r | j j |i |��S | j j |i |��j � S r )r r �lsr/ r3 r4 s r r> zDaskWorkerFileSystem.ls] sF � � �;�;� �4�7�7�:�:�t�.�v�.� .� �4�8�8�;�;��/��/�7�7�9� 9r c
� � | j r" | j j |f||||d�|��S t d| |||||d�|��S )N)�mode�
block_size�
autocommit�
cache_options)r r) r@ rA rB rC r )r r �_open�DaskFile)r" r) r@ rA rB rC r# s r rD zDaskWorkerFileSystem._openc sp � � �;�;� �4�7�7�=�=����%�%� +� � �
�
� � ����%�%� +�
� ��
r c � � | j rA| j ||� 5 }|j |� |j ||z
� cd d d � S | j j ||||� j
� S # 1 sw Y y xY wr )r rD �seek�readr/ � fetch_ranger3 )r" r) r@ �start�end�fs r rI z DaskWorkerFileSystem.fetch_range� sm � � �;�;����D�$�'�1����u�
��v�v�c�E�k�*�
+�
+� �8�8�'�'��d�E�3�?�G�G�I� I� (�'�s �%A:�:B)NNNN)�rbNTN)�__name__�
__module__� __qualname__�__doc__r � staticmethodr+ r! r2 r7 r: r< r> rD rI �
__classcell__�r$ s @r r r s_ �� �� JN�!� �� ��
*�6� 3� 5� 3� :� ����
�:Jr r c �4 � � e Zd Zd� fd� Zdd�Zd� Zd� Z� xZS )rE c �D �� |dk7 r t d� �t �| � di |�� y )NrM z1Remote dask files can only be opened in "rb" moder )r r r )r" r@ r# r$ s �r r zDaskFile.__init__� s& �� � �4�<� � P�Q� Q�
���"�6�"r c � � y r r )r" �finals r �
_upload_chunkzDaskFile._upload_chunk� s � � r c � � y)zCreate remote file/uploadNr r0 s r �_initiate_uploadzDaskFile._initiate_upload� s � � r c �f � | j j | j | j ||� S )z*Get the specified set of bytes from remote)r rI r) r@ )r" rJ rK s r � _fetch_rangezDaskFile._fetch_range� s% � ��w�w�"�"�4�9�9�d�i�i���D�Dr )rM )F)rN rO rP r rY r[ r] rS rT s @r rE rE � s �� �#�
�
�Er rE )r- �distributed.clientr r �distributed.workerr �fsspecr � fsspec.specr r � fsspec.utilsr
r r r rE r r r �<module>rc sB �� � 9� %� � @� .� �#�nJ�-� nJ�bE�#� Er
