-Ph=dZddlZddlZddlZddlZddlZddlZddlmZddl m Z ddl m Z Gddej ZGd d eZGd d eZGd dZdS)z Post-processing hooks N)ZipFile)TarFile) get_loggerceZdZdZddZeejdZejdZ ejdZ dZ dS) ExtractorProcessora Abstract base class for extractions from compressed archives. Subclasses can be used with :meth:`pooch.Pooch.fetch` and :func:`pooch.retrieve` to unzip a downloaded data file into a folder in the local data store. :meth:`~pooch.Pooch.fetch` will return a list with the names of the extracted files instead of the archive. Parameters ---------- members : list or None If None, will unpack all files in the archive. Otherwise, *members* must be a list of file names to unpack from the archive. Only these files will be unpacked. extract_dir : str or None If None, files will be unpacked to the default location (a folder in the same location as the downloaded zip file, with a suffix added). Otherwise, files will be unpacked to ``extract_dir``, which is interpreted as a *relative path* (relative to the cache location provided by :func:`pooch.retrieve` or :meth:`pooch.Pooch.fetch`). Nc"||_||_dSN)members extract_dir)selfr r s P/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/pooch/processors.py__init__zExtractorProcessor.__init__/s &cdS)z String appended to unpacked archive folder name. Only used if extract_dir is None. MUST BE IMPLEMENTED BY CHILD CLASSES. Nr s rsuffixzExtractorProcessor.suffix3rcdS)zf Return all the members in the archive. MUST BE IMPLEMENTED BY CHILD CLASSES. Nr)r fnames r _all_memberszExtractorProcessor._all_members<rrcdS)z This method receives an argument for the archive to extract and the destination path. MUST BE IMPLEMENTED BY CHILD CLASSES. Nr)r rr s r _extract_filez ExtractorProcessor._extract_fileCrrc  j|jz_nV|tjjdd}tj|j_jjs|}nj}|dvs?tj jrtfd|Ds6tj jd |jg}tj jD]\}}} | D]} tjtjtj|j|  j t! fd jDr3|tj|| |S) a Extract all files from the given archive. Parameters ---------- fname : str Full path of the zipped file in local storage. action : str Indicates what action was taken by :meth:`pooch.Pooch.fetch` or :func:`pooch.retrieve`: * ``"download"``: File didn't exist locally and was downloaded * ``"update"``: Local file was outdated and was re-download * ``"fetch"``: File exists and is updated so it wasn't downloaded pooch : :class:`pooch.Pooch` The instance of :class:`pooch.Pooch` that is calling this. Returns ------- fnames : list of str A list of the full path to all files in the extracted archive. Nr)maxsplitrupdatedownloadc3K|]F}tjtjj|VGdSr )ospathexistsjoinr ).0mr s r z.ExtractorProcessor.__call__..ssUFGrw||D,.sR//@AG&&rw'7'7':':;;//////r)r rrsplitr!r"sepr$r rr#allmakedirsrwalkr+r,anyappend) r ractionpooch archive_dirr fnamesr"_filesfilenamer,s ` @r__call__zExtractorProcessor.__call__Ks2   #$t{2D  ,,rw{Q,??BK!w||K9IJJD  < t| ''..GGlG - - -GNN4#344.KR. K(4 8 8 8 8   ud&6 7 7 7 gd&677 @ @ND!U! @ @'**GLLt7G!H!H(SS<'3////EI\///,,'MM"',,tX">">??? @ r)NN) __name__ __module__ __qualname____doc__rpropertyabcabstractmethodrrrr;rrrrrs.''''  X         =====rrc4eZdZdZedZdZdZdS)Unzipa Processor that unpacks a zip archive and returns a list of all files. Use with :meth:`pooch.Pooch.fetch` or :func:`pooch.retrieve` to unzip a downloaded data file into a folder in the local data store. The method/function will return a list with the names of the unzipped files instead of the zip archive. The output folder is ``{fname}.unzip``. Parameters ---------- members : list or None If None, will unpack all files in the zip archive. Otherwise, *members* must be a list of file names to unpack from the archive. Only these files will be unpacked. extract_dir : str or None If None, files will be unpacked to the default location (a folder in the same location as the downloaded zip file, with the suffix ``.unzip`` added). Otherwise, files will be unpacked to ``extract_dir``, which is interpreted as a *relative path* (relative to the cache location provided by :func:`pooch.retrieve` or :meth:`pooch.Pooch.fetch`). cdS)l String appended to unpacked archive folder name. Only used if extract_dir is None. z.unziprrs rrz Unzip.suffix xrc~t|d5}|cdddS#1swxYwYdS)(Return all members from a given archive.rN)rnamelist)r rzip_files rrzUnzip._all_memberss UC  'H$$&& ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' 's 266ct|d5}|j:td||||ne|jD]]td||fd|D}|||^ddddS#1swxYwYdS)o This method receives an argument for the archive to extract and the destination path. rJNz"Unzipping contents of '%s' to '%s'r"!Extracting '%s' from '%s' to '%s'cg|]S}tj|tjQ|TSr)r!r"r+r*)r%namemembers r z'Unzip._extract_file..s]&&& 7++D11<r?r@rrrrrrrDrDs\4X''' RRRRRrrDc4eZdZdZedZdZdZdS)Untara Processor that unpacks a tar archive and returns a list of all files. Use with :meth:`pooch.Pooch.fetch` or :func:`pooch.retrieve` to untar a downloaded data file into a folder in the local data store. The method/function will return a list with the names of the extracted files instead of the archive. The output folder is ``{fname}.untar``. Parameters ---------- members : list or None If None, will unpack all files in the archive. Otherwise, *members* must be a list of file names to unpack from the archive. Only these files will be unpacked. extract_dir : str or None If None, files will be unpacked to the default location (a folder in the same location as the downloaded tar file, with the suffix ``.untar`` added). Otherwise, files will be unpacked to ``extract_dir``, which is interpreted as a *relative path* (relative to the cache location provided by :func:`pooch.retrieve` or :meth:`pooch.Pooch.fetch`). cdS)rFz.untarrrs rrz Untar.suffixrGrctj|d5}d|DcdddS#1swxYwYdS)rIrJcg|] }|j Sr)rR)r%rVs rrTz&Untar._all_members..s@@@$DI@@@rN)ropen getmembers)r rtar_files rrzUntar._all_memberss \% % % A@@(*=*=*?*?@@@ A A A A A A A A A A A A A A A A A AsAAActj|d5}|j:td||||ne|jD]]td||fd|D}|||^ddddS#1swxYwYdS)rNrJNz"Untarring contents of '%s' to '%s'rOrPcg|]X}tj|jtjV|YSr)r!r"r+rRr*)r%rVrSs rrTz'Untar._extract_file..sd&&& 7++DI66AAG,,V44&&&&rrU)rr_r rrVrWr`)r rr rarXrSs @rrzUntar._extract_files\ \% % % R|# !!8%###5555"lRRFLL%%;VUK&&&&$,$7$7$9$9&&&N''['QQQQ9 R R R R R R R R R R R R R R R R R RsB'C  CCNrYrrrr[r[s_4XAAA !R!R!R!R!Rrr[cBeZdZdZdeeeedZddddZd d Z d Z d Z dS) Decompressa3 Processor that decompress a file and returns the decompressed version. Use with :meth:`pooch.Pooch.fetch` or :func:`pooch.retrieve` to decompress a downloaded data file so that it can be easily opened. Useful for data files that take a long time to decompress (exchanging disk space for speed). Supported decompression methods are LZMA (``.xz``), bzip2 (``.bz2``), and gzip (``.gz``). File names with the standard extensions (see above) can use ``method="auto"`` to automatically determine the compression method. This can be overwritten by setting the *method* argument. .. note:: To unpack zip and tar archives with one or more files, use :class:`pooch.Unzip` and :class:`pooch.Untar` instead. The output file is ``{fname}.decomp`` by default but it can be changed by setting the ``name`` parameter. .. warning:: Passing in ``name`` can cause existing data to be lost! For example, if a file already exists with the specified name it will be overwritten with the new decompressed file content. **Use this option with caution.** Parameters ---------- method : str Name of the compression method. Can be "auto", "lzma", "xz", "bzip2", or "gzip". name : None or str Defines the decompressed file name. The file name will be ``{fname}.decomp`` if ``None`` (default) or the given name otherwise. Note that the name should **not** include the full (or relative) path, it should be just the file name itself. N)autolzmaxzgzipbzip2rgrirj)z.xzz.gzz.bz2rfc"||_||_dSr )methodrR)r rlrRs rrzDecompress.__init__Ns  rc:|j|dz}nBtjtj||j}|dvstj|st d|||j| |}t|d5}| |5}tj ||dddn #1swxYwYdddn #1swxYwY|S)aK Decompress the given file. The output file will be either ``{fname}.decomp`` or the given *name* class attribute. Parameters ---------- fname : str Full path of the compressed file in local storage. action : str Indicates what action was taken by :meth:`pooch.Pooch.fetch` or :func:`pooch.retrieve`: - ``"download"``: File didn't exist locally and was downloaded - ``"update"``: Local file was outdated and was re-download - ``"fetch"``: File exists and is updated so it wasn't downloaded pooch : :class:`pooch.Pooch` The instance of :class:`pooch.Pooch` that is calling this. Returns ------- fname : str The full path to the decompressed file. Nz.decomprz-Decompressing '%s' to '%s' using method '%s'.zw+b) rRr!r"r$dirnamer#rrVrl_compression_moduler_shutil copyfileobj)r rr4r5 decompressedmoduleoutput compresseds rr;zDecompress.__call__Rs|6 9  9,LL7<<(>(> JJL + + +27>>,3O3O + LL  ?     --e44FlE** ;f[['';:&z6:::;;;;;;;;;;;;;;; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ;s6DC9- D9C= =DC= DDDcXd}|j|jvrad|jdt|jd}|jdvrd||g}t ||jdkrt j|d}||j vrWd |dt|j d}|d vrd||g}t ||j|j |S|j|jS) a Get the Python module compatible with fname and the chosen method. If the *method* attribute is "auto", will select a method based on the extension. If no recognized extension is in the file name, will raise a ValueError. z:To unpack zip/tar archives, use pooch.Unzip/Untar instead.zInvalid compression method 'z'. Must be one of 'z'.>tarzip rfzUnrecognized file extension '>.tar.zip) rlmoduleslistkeysr$ ValueErrorr!r"splitext extensions)r rerror_archivesmessageexts rrozDecompress._compression_module~s_V ;dl * *At{AA#' (9(9(;(;#<#<AAA {n,,((G^#<==W%% % ;& '""5))"-C$/))HCHH'+DO,@,@,B,B'C'CHHH***!hh'@AAG )))< 45 5|DK((r)rfN) r<r=r>r?rgribz2r}rrr;rorrrreresv))VTtcRRG@@J***X)))))rre)r?rAr!rrirgrpzipfilertarfilerutilsrABCrrDr[rerrrrsf qqqqqqqqhERERERERER ERERERPIRIRIRIRIR IRIRIRX{){){){){){){){){){)r