M/Phy(ddlmZddlmZddlmZmZddlmZm Z m Z m Z m Z ddl Z ddlmZmZddlmZddlmZddlZdd lmZmZmZd!d ZGd deZd"dZdZdZ dZ!dZ"dZ#d#dZ$dZ%d$dZ&d%dZ'd%dZ(d%dZ)dZ*d&d Z+dS)')lrange)StringIO)environmakedirs)abspathdirnameexists expanduserjoinN) HTTPErrorURLError)urljoin)urlopen)Indexread_csv read_stata%https://www.stata-press.com/data/r11/TcFt||dz}t|S)a Download and return an example dataset from Stata. Parameters ---------- data : str Name of dataset to fetch. baseurl : str The base URL to the stata datasets. as_df : bool Deprecated. Always returns a DataFrame Returns ------- dta : DataFrame A DataFrame containing the Stata dataset. Examples -------- >>> dta = webuse('auto') Notes ----- Make sure baseurl has trailing forward slash. Does not do any error checking in response URLs. z.dta)rr)databaseurlas_dfurls Z/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/datasets/utils.pywebusers#6 '4; ' 'C c??ceZdZdZdZdS)Datasetc d|_d|_d|_d|_t||||_ |jt|_ dS#YdSxYwN) endogexogrnamesdict__init____dict__astypefloatraw_data)selfkws rr$zDataset.__init__/si     dB   I,,U33DMMM  DDs $A&&A+c*t|jSr)str __class__)r)s r__repr__zDataset.__repr__?s4>"""rN)__name__ __module__ __qualname__r$r.rrrr.s2 #####rrcj|j}t|tr^||}||}|||gd}n|||}n|jdd|f}t |j}|||d}nVt|tr!|||}n |||}|Yt|jdd|f}||_ ||_ | ||}t |j} t|t ||||| } | S)N)axis)rr"r r! endog_name exog_name) columns isinstanceintcopydroploclistrilocindex set_indexr) r endog_idxexog_idx index_idxr"r6r r!r@r7datasets rprocess_pandasrFCs LE)S!!09% Z %%''  99j\922DDh(--//DDI&++--%-((  99Za900DD # & & 0h(--//DDh(--//Ddi9 -.. ZZ\\ ~~eI.//T\""I4tE{{%J)MMMG Nrc |jttdt |dzr|d}|S)z All the Rdatasets have the integer row.labels from R if there is no real index. Strip this for a zero-based index r4T)r<)r@equalsrrlen reset_index)rs r_maybe_reset_indexrKcsS  zvaTQ778899+T** KrcZ|durd}n#|durtd}nt|}|S)NFT) get_data_home)caches r _get_cacherOms< ~~ $d##e$$ Lrcddl}t|d5}|||ddddS#1swxYwYdS)Nrwb)zlibopenwritecompress)r cache_pathrRzfs r _cache_itrXxsKKK j$  &2 t$$%%%&&&&&&&&&&&&&&&&&&s)A  AAcddl}t|d5}||cdddS#1swxYwYdS)Nrrb)rRrS decompressread)rVrRrWs r _open_cacher]~sKKK j$  *2rwwyy))******************s'A  A A cd}||dddd}|d}t|dkr|d xxd z cc<n|d xxd z cc<d|d z}t||} t |}d }n#YnxYw|s5t |d}|t||||fS)z Tries to load data from cache location otherwise downloads it. If it downloads the data and cache is not None then it will put the downloaded data in the cache path. FNz:///,.r4z-v2rz.zipT)timeout)splitreplacerIr r]rr\rX)rrN from_cache file_namerVrs r_urlopen_cachedrjs J IIe$$R(00c:: OOC(( y>>A   bMMMU "MMMM aLLLE !LLLHHY''&0 %++  z**DJJ  D (sA&&&++--   dJ ' ' '  s &B88B<csvc||dz|zz} t||\}}n7#t$r*}dt|vrtd|z|d}~wwxYw|dd}t ||fS)Nz.%s404zDataset %s was not found.utf-8strict)rjr r, ValueErrordecoder)base_urldatanamerN extensionrrrherrs r _get_datarvs h&)3 3C*366jj  CHH  88CDD DI  ;;w ) )D D>>: %%s! A%AAc xd}t||\}}|dd}tt|}t j|j|k|j|k}|std|d|d|d|j |}|dj d S) NzRhttps://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/datasets.csvrnrozItem z from Package z& was not found. Check the CSV file at z to verify the Item and Package.Titler) rjrqrrnp logical_andItemPackageanyrpr=r?) rspackagerN index_urlr_r@idx dataset_metas r_get_dataset_metars1Ii//GD! ;;w ) )D Xd^^ $ $E .x/'1I J JC 7799  KH K KG K K( K K K   9S>L  %a ((rdatasetsFc@d|zdz}d|zdz}t|}t|||\}}t|d}t|}t |||}t|||d\}} t |||||S) a7download and return R dataset Parameters ---------- dataname : str The name of the dataset you want to download package : str The package in which the dataset is found. The default is the core 'datasets' package. cache : bool or str If True, will download this data into the STATSMODELS_DATA folder. The default location is a folder called statsmodels_data in the user home folder. Otherwise, you can specify a path to a folder to use for caching the data. If False, the data will not be cached. Returns ------- dataset : Dataset A `statsmodels.data.utils.Dataset` instance. This objects has attributes: * data - A pandas DataFrame containing the data * title - The dataset title * package - The package from which the data came * from_cache - Whether not cached data was retrieved * __doc__ - The verbatim R documentation. Notes ----- If the R dataset has an integer index. This is reset to be zero-based. Otherwise the index is preserved. The caching facilities are dumb. That is, no download dates, e-tags, or otherwise identifying information is checked to see if the data should be downloaded again or not. If the dataset is in the cache, it's used. zJhttps://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/csv/r`zJhttps://raw.githubusercontent.com/vincentarelbundock/Rdatasets/master/doc/z/rst/r) index_colrst)r__doc__r~titlerh)rOrvrrKrrr\) rsr~rN data_base_url docs_base_urlrrhrdocrs r get_rdatasetrsJ##*++./M##*++23M u  E %@@D* DA & & &D d # #D h 7 7E }hu = =FC chhjj'( * * **rc|#tjdtdd}t|}t |st ||S)a Return the path of the statsmodels data dir. This folder is used by some large dataset loaders to avoid downloading the data several times. By default the data dir is set to a folder named 'statsmodels_data' in the user home folder. Alternatively, it can be set by the 'STATSMODELS_DATA' environment variable or programatically by giving an explicit folder path. The '~' symbol is expanded to the user home folder. If the folder does not already exist, it is automatically created. NSTATSMODELS_DATA~statsmodels_data)rgetr r r r data_homes rrMrMs]K 2 $S*< = =?? 9%%I )   rcLt|}tj|dS)z.Delete all the content of the data home cache.N)rMshutilrmtreers rclear_data_homers%i((I M)rcd|dn|} t|n#t$r }Yd}~dSd}~wwxYwdS)zCheck if internet is availableNzhttps://github.comFT)rr )rrus rcheck_internetrsT"%+  3C uuuuu 4s  --c4g}|D]}|dr |dr |dd}n?|dr |dd}n|dr |dd}||||_|S)a  Remove leading and trailing single quotes Parameters ---------- df : DataFrame DataFrame to process Returns ------- df : DataFrame DataFrame with stripped column names Notes ----- In-place modification 'r4r_N) startswithendswithappendr8)dfr8cs rstrip_column_namesr"s$G  <<   !**T"2"2 !B$AA \\$   !""AA ZZ   #2#AqBJ Irractt|}t||}|dkrdnd}i}|dkrddi}t|f||d|}|r|t }|S)zStandard simple csv loaderrapythonrfloat_precisionhigh)sepengine)rrr rr&r') base_filecsv_namer convert_floatfilepathfilenamerrrs rload_csvrAswy))**HHX&&HXXFO }},f5 H H#f H H H HD"{{5!! Kr)rT)rNN)rk)rFr)raF),statsmodels.compat.pythonriorosrros.pathrrr r r r urllib.errorr r urllib.parserurllib.requestrnumpyrypandasrrrrr#rrFrKrOrXr]rjrvrrrMrrrrr2rrrs,,,,,, >>>>>>>>>>>>>> ,,,,,,,, """"""..........>#####d###*@&&& ***< & & & &)))$2*2*2*2*n0 >      r