Q/Ph ZdZddlZddlZddlZddlZddlmZddlmZm Z ddl Z ddl m Z deddfdZd edefd Zd ed edefd ZdedefdZdede jfdZde jde jfdZdde jdede jfdZdZde jdefdZddededefdZdede edefdZdS) zUtils for pandas DataFrames.N)Path)AnyOptional)hash_pandas_object extensionreturnc6tjd|ddS)zyWarn the user when an extension is not supported. Args: extension: The extension that is not supported. z3There was an attempt to read a file with extension a[, we assume it to be in CSV format. To prevent this warning from showing up, please rename the file to any of the extensions supported by pandas (docs: https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html) If you think this extension should be supported, please report this as an issue: https://github.com/ydataai/ydata-profiling/issuesN)warningswarn)rs _/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/ydata_profiling/utils/dataframe.py warn_readr s8  M 5  5 5 5file_extensionc.|dvS)uBDetermine if the given file extension indicates a compression format that pandas can handle automatically. Args: file_extension (str): the file extension to test Returns: bool: True if the extension indicates a compression format that pandas handles automatically and False otherwise Notes: Pandas can handle on the fly decompression from the following extensions: ‘.bz2’, ‘.gz’, ‘.zip’, or ‘.xz’ (otherwise no decompression). If using ‘.zip’, the ZIP file must contain exactly one data file to be read in. )z.bz2z.gzz.xzz.zip)lower)rs r is_supported_compressionrs    ! !%C CCrtextsuffixcd|r-||r|dt| n|S)aRemoves the given suffix from the given string. Args: text (str): the string to remove the suffix from suffix (str): the suffix to remove from the string Returns: str: the string with the suffix removed, if the string ends with the suffix, otherwise the unmodified string Notes: In python 3.9+, there is a built-in string method called removesuffix() that can serve this purpose. N)endswithlen)rrs r remove_suffixr,s6$* MdmmF.C.C M43v;;,  Mr file_namec|j}t|rAtt t ||jn|S)aReturns the uncompressed extension of the given file name. Args: file_name (Path): the file name to get the uncompressed extension of Returns: str: the uncompressed extension, or the original extension if pandas doesn't handle it automatically )rrrrrstr)rrs r uncompressed_extensionr<s^ &&((I $I . . ]3y>>//119 = =>>EE rct|}|dkr#tjt|}n|dkr%tjt|d}n_|dkr#tjt|}n6|dkr%tjt|d}n |d vr"tjt|}n|d vr"tjt|}n|d vr"tjt|}n|d kr"tj t|}nq|d vr"tj t|}nK|dkrtd|dkrt|tjt|}|S)a Read DataFrame based on the file extension. This function is used when the file is in a standard format. Various file types are supported (.csv, .json, .jsonl, .data, .tsv, .xls, .xlsx, .xpt, .sas7bdat, .parquet) Args: file_name: the file to read Returns: DataFrame Notes: This function is based on pandas IO tools: https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html https://pandas.pydata.org/pandas-docs/stable/reference/io.html This function is not intended to be flexible or complete. The main use case is to be able to read files without user input, which is currently used in the editor integration. For more advanced use cases, the user should load the DataFrame in code. z.jsonz.jsonlT)linesz.dtaz.tsv )sep)z.xlsz.xlsx)z.hdfz.h5)z .sas7bdatz.xptz.parquet)z.pklz.picklez.tarzTtar compression is not supported directly by pandas, please use the 'tarfile' modulez.csv) rpd read_jsonr read_stataread_csv read_excelread_hdfread_sas read_parquet read_pickle ValueErrorr )rrdfs r read_pandasr,Ms&'y11IG \#i.. ) ) h   \#i.. 5 5 5 f   ]3y>> * * f   [YT 2 2 2 ' ' ' ]3y>> * * o % % [Y ( ( + + + [Y ( ( j _S^^ , , ) ) ) ^C NN + + f   b       i [Y ( ( Irr+c|ddidd|jjvr d|jjD|j_|S)aLIf the DataFrame contains a column or index named `index`, this will produce errors. We rename the {index,column} to be `df_index`. Args: df: DataFrame to process. Returns: The DataFrame with {index,column} `index` replaced by `df_index`, unchanged if the DataFrame does not contain such a string. indexdf_indexTcolumnsinplacec"g|] }|dkr|nd S)r.r/).0xs r z rename_index..s$TTTqG||!!TTTr)renamer.names)r+s r rename_indexr:sNIIw +TI:::"(.  TTRX^TTT Irtypesctttg|jD]}||fd}|rtj|| }| |dz}t|}| |gdtj ||gd}|S)aExpand non-nested lists, dicts and tuples in a DataFrame into columns with a prefix. Args: types: list of types to expand (Default: list, dict, tuple) df: DataFrame Returns: DataFrame with the dict values as series, identifier by the combination of the series and dict keys. Notes: TODO: allow for list expansion by duplicating rows. TODO: allow for expansion of nested data structures. Nc^t|votfd|D S)Nc3:K|]}t|vVdSN)type)r5yr;s r z1expand_mixed....s.7T7TQQ58H7T7T7T7T7T7Tr)r@any)r6r;s r zexpand_mixed..s8477e+TC7T7T7T7TRS7T7T7T4T4T0Tr_Tr0)axis)listdicttupler1dropnamapallr! DataFrametolist add_prefix expand_mixeddropconcat)r+r; column_namenon_nested_enumerationexpandeds ` r rQrQs }tU#z33  {O VXX STTTT U U  " % % ' ' 3|B{O$:$:$<$<$C$C$E$EFFH**;+<==H$H--H GG[M4G 8 8 8B>222B Irz2@cdt|jt}t j|d}t|S)zHash a DataFrame (implementation might change in the future) Args: df: the DataFrame Returns: The DataFrame's hash  zutf-8) joinrvaluesastyperhashlibsha256encode hexdigest HASH_PREFIX)r+ hash_valuesdigests r hash_dataframercsi)).r229@@EEFFK ^K..w77 8 8 B B D DF #6 # ##rFvalue allow_unicodecjt|}|rtjd|}n.QqT]]__r)keydescc6|dSryrzr|s r rDz#sort_column_names..r}rT)r~reversez3"sort" should be "ascending", "descending" or None.)r startswithrIsorteditemsr*)rtrus r sort_column_namesrs | ::<rs$""  777777       DS DT D D D D N NS NS N N N N ds"/4/BL////dR\bl"%%R\%#%%%%%R $r|$$$$$$5535t55555* 4 x}       r