Q/PhOVddlZddlZddlZddlmZddlmZmZmZddl m Z ej 5ej dddl Z dddn #1swxYwYe sddlmZedZnddlmZdd lmZmZddlZddlZdd lmZdd lmZdd lmZdd lm Z m!Z!m"Z"ddl#m$Z$ddl%m&Z&ddl'm(Z(ddl)m*Z+ddl,m-Z-ddl.m/Z/m0Z0m1Z1m2Z2ddl3m4Z4ddl5m6Z6ddl7m8Z8ddl9m:Z:ddl;mZ>ddl?m@Z@ddlAmBZBe@dZCeGddez)ProfileReport.__init__..s&44\Yy444444T)exclude_defaultsr)_ProfileReport__validate_inputstype_df_typer$r from_file isinstancepdr ranyupdaterget_arg_groupsdictlen shorthandsvars timeseriesactiver-$_ProfileReport__initialize_dataframer*r)_df_hash_sample _type_schema_typeset _summarizerreport)selfr*r+r,r-r.r/r0r1r2r3r4r)r5kwargs report_configgroupscfgr;keyrLr<s r=__init__zProfileReport.__init__DsIR r7FKFFFR  0' 0 @()>?? $JJ00==MM  "MM"bl++ 0 ( - - (  $  44V444 4 4 R**C"( A A 3A**V%:3%?%?@@C)0041P1PQQM v;;??!'!26!:!: J)00 !!*--22D2IIM  9)0088M/5 %,  :f :39M  ) 0--b-@@#   ' %  AAA  r?returnc8||std||rtdt|tjr||jrtddSdS|rt d|(|jrtddSdS)Nz3Can init a not-lazy ProfileReport with no DataFramez=Arguments `config_file` and `minimal` are mutually exclusive.z8DataFrame is empty. Pleaseprovide a non-empty DataFrame.zFTime-Series dataset analysis is not yet supported for Spark DataFrames) ValueErrorrErFr emptyNotImplementedErrorrddisEmpty)r*r+r,r1r2s r=__validate_inputszProfileReport.__validate_inputss :d:RSS S  "w "O  b", ' ' ~"(~ Q~~~  )\ 26>>#3#3 Qr?rYct||jjj|t |t jr|jjjr}|jjjrX| |jjj}| |jjjd}d|j _ n| }|S)N)r*rN)byF)drop)loggerinfo_def_reportrMrNrOrErFr r- sort_values set_indexindexr& sort_index)r*rYs r=__initialize_dataframez$ProfileReport.__initialize_dataframes $)4;    N2r|,, "-4 !,3 %^^}'9'D'K^LL\\-"4"?"FU\SS $ ]]__ r?subsetc||dvrtd||dvrd|_d|_d|_||dkrd|_| d|_dSdS)aTInvalidate report cache. Useful after changing setting. Args: subset: - "rendering" to invalidate the html, json and widget report rendering - "report" to remove the caching of the report structure - None (default) to invalidate all caches Returns: None N) renderingrVz:'subset' parameter should be None, 'rendering' or 'report'rV)r`_widgets_json_html_report_description_set)rWrps r=invalidate_cachezProfileReport.invalidate_caches  &0G"G"GL  >V'>>> DMDJDJ >Vx//DL >$(D ! ! ! >r?c\|jt|j|j|_|jSr8)rTrr)rSrWs r=r3zProfileReport.typesets( = ,T[$:KLLDM}r?c|j2d}|jtjurd}t |j||_|jS)NFT) use_spark)rUrCrFr rr3)rWr|s r=r4zProfileReport.summarizersD   #I}BL00 24<9UUUD r?c|j1t|j|j|j|j|j|_|jSr8)rw describe_dfr)r*r4r3rRrzs r=description_setzProfileReport.description_set sA  ($/   %%D !$$r?c^|j |jt|j|_|jSr8)rQr*r"rzs r=df_hashzProfileReport.df_hashs* = TW%8*4733DM}r?c\|jt|j|j|_|jSr8)rvrr)rrzs r=rVzProfileReport.reports( < / T=QRRDL|r?cP|j||_|jSr8)ru _render_htmlrzs r=htmlzProfileReport.html#% : **,,DJzr?cP|j||_|jSr8)rt _render_jsonrzs r=jsonzProfileReport.json)rr?ct|jjdtr2t |jjddkrt d|j||_|jS)Nnz[Widgets interface not (yet) supported for comparing reports, please use the HTML rendering.)rErtablelistrK RuntimeErrorrs_render_widgetsrzs r=widgetszProfileReport.widgets/sy t+1#6 = = D(.s344q88m  =  0022DM}r?c|jjS)zGet duplicate rows and counts based on the configuration Returns: A DataFrame with the duplicate rows and their counts. )r duplicatesrzs r=get_duplicateszProfileReport.get_duplicates=s #..r?c|jjS)z~Get head/tail samples based on the configuration Returns: A dict with the head and tail samples. )rr0rzs r= get_samplezProfileReport.get_sampleEs #**r?c|jS)zReturn the description (a raw statistical summary) of the dataset. Returns: Dict containing a description for each variable in the DataFrame. )rrzs r=get_descriptionzProfileReport.get_descriptionMs ##r?c.d|jjDS)zGet variables that are rejected for analysis (e.g. constant, mixed data types) Returns: a set of column names that are unsupported cFh|]}|jtjk|jSr9) alert_typerREJECTED column_name)r:alerts r= z7ProfileReport.get_rejected_variables..[s6   9#555  555r?)ralertsrzs r=get_rejected_variablesz$ProfileReport.get_rejected_variablesUs)   -4    r? output_filesilentctj5tjdtjdj}dddn #1swxYwYt tt| d}|dkrtj dt|tstt|}|jdkr|}n|jjjsot|j|jj_|jjj&t|jdz|jj_t/|j||}|jd kr4|j}|d }tj d |d |jj }t7d d |5}||d|dddn #1swxYwY|s ddlm} | |!"dS#tF$rAddl$} | %|!"YdSwxYwdS)aWrite the report to a file. Args: output_file: The name or the path of the file to generate including the extension (.html, .json). silent: if False, opens the file in the default browser or download it in a Google Colab environment rPillowN.) rzGTry running command: 'pip install --upgrade Pillow' to avoid ValueErrorz.json_assetsz.htmlz Extension zh not supported. For now we assume .html was intended. To remove this warning, please use .html or .json.rzExport report to filetotaldescdisablezutf-8)encodingrfiles)&warningscatch_warnings simplefilter pkg_resourcesget_distributionversiontuplemapintsplitwarnrErstrsuffixto_jsonr)rinlineparent assets_path assets_prefixstemr to_html with_suffix progress_barr write_textrH google.colabrdownloadabsoluteas_uriModuleNotFoundError webbrowser open_new_tab) rWrrpillow_version version_tupledatardisable_progress_barpbarrrs r=to_filezProfileReport.to_fileas[ $ & & N N  !( + + +*;HEEMN N N N N N N N N N N N N N N Nc#~';';C'@'@AABB 9 $ $ MY   +t,, 1s;//00K   ( (<<>>DD;#* =/2;3E/F/F  ,;#19589I5J5JY5VDK$2"4; <<<<<>>D!W,,$+)55g>>  JJJJ $(;#;; 1;O       " "4' " : : : KKMMM                  I I......{3355<<>>?????& I I I!!!!'' (<(<(>(>(E(E(G(GHHHHHH I  I Is6.AAA+,H##H'*H'1?I22AJ=<J=cddlm}|j}tdd|jj 5}|t j||jj j |jj j |jj j |jj j |jj jjd|jj jj|jj jj|jjj|jjj|jjd }|jj jrddlm}||d d }|dddn #1swxYwY|S) Nr) HTMLReportrz Render HTMLrydata_profiling_version) navofflinerr primary_colorlogothemetitledater)minifyT)remove_all_empty_spaceremove_comments),ydata_profiling.report.presentation.flavoursrrVrr)rcopydeepcopyrenderr navbar_showuse_local_assetsrrstyleprimary_colorsrrranalysisr date_startpackage minify_html htmlmin.mainrrH)rWrrVrrrs r=rzProfileReport._render_htmlsKKKKKK -T[5M1M     :dmF3344;;K$0 (9{'."k.<"k.4CAF[%+0k&,2*39)2=,45NO<  D{+ W//////vd4QUVVV KKMMM)               * sD EEEcddlm}|j}tdd|jj d5}|t j|}| dddn #1swxYwY|S)Nr) WidgetReportrzRender widgetsF)rrrleave) rrrVrr)rrrrrH)rWrrVrrs r=rzProfileReport._render_widgetssMMMMMM ! 00      "l4=#8#899@@BBG KKMMM               sAA<<BBcZdtdtffd |j}tdd|jj 5}t |}|}t ||j}tj|d}| dddn #1swxYwY|S) Nor^ct|rt|}t|tr fd|DSt|t t ttfr|St|trfd|DSt|trfd|DSt|tj r| St|tjr|dSt|t jr|St|t&r|St|t jr|St|S)Nc@i|]\}}||Sr9r9)r:kv encode_its r= zAProfileReport._render_json..encode_it..s/IIItq! ! iillIIIr?c&g|] }|Sr9r9r:rrs r= zAProfileReport._render_json..encode_it..!444QIIaLL444r?c&h|] }|Sr9r9rs r=rz@ProfileReport._render_json..encode_it..rr?records)orient)r r rErJitemsboolrfloatrrsetrFSeriesto_listr to_dictnpndarraytolistrgenericitem)rrs r=rz-ProfileReport._render_json..encode_itsA 1II!T"" "IIIIqwwyyIIIIa$UC!899"H4(("4444!44443'' "4444!444429-- "$9QYY[[1112<00 "$9QYYiY%@%@AAA2:.."$9QXXZZ0006**"$9QVVXX...2:.."6688Oq66Mr?rz Render JSONr)indent) rrrr)rrrrdumpsrH)rW descriptionrdescription_dictrrs @r=rzProfileReport._render_jsons  " " " " " " " "2* -T[5M1M     -k:: (y)9:: -.> LL :.q999D KKMMM                sAB  B$'B$c|jS)zGenerate and return complete template as lengthy string for using with frameworks. Returns: Profiling report html including wrapper. )rrzs r=rzProfileReport.to_htmls yr?c|jS)z_Represent the ProfileReport as a JSON string Returns: JSON string )rrzs r=rzProfileReport.to_jsons yr?cddlm}ddlm}t j5t jd|||j|ddddS#1swxYwYdS)aUsed to output the HTML representation to a Jupyter notebook. When config.notebook.iframe.attribute is "src", this function creates a temporary HTML file in `./tmp/profile_[hash].html` and returns an Iframe pointing to that contents. When config.notebook.iframe.attribute is "srcdoc", the same HTML is injected in the "srcdoc" attribute of the Iframe. Notes: This constructions solves problems with conflicting stylesheets and navigation links. rdisplay)get_notebook_iframerN)IPython.displayrrFsR ''''''''''>>>>>>XH(###4&&JJ333333,,,,,,,,!!!!!!""""""BBBBBBBBBBBBBBBB111111222222BBBBBB////// ;:::::777777999999=<<<<<::::::888888222222 n - - - FUFUFUFUFUO%7FUFU FUFUFUsAAA