M/PhA dZddlmZddlZddlmZGddZedkrddl m Z dd l m Z d%d Z d&dZeddZdZdZedeeeZededeedgdeedeedeedeegde jgdjgddeeddeed d!eed gdgdZe jd"d#gejejZeedd!geed$e jgdejejZeegdeegddeeegddj  dZ!e!rBddl"m#Z$e%de jd"j&e$j'dSdSdS)'a}Helper class for Monte Carlo Studies for (currently) statistical tests Most of it should also be usable for Bootstrap, and for MC for estimators. Takes the sample generator, dgb, and the statistical results, statistic, as functions in the argument. Author: Josef Perktold (josef-pktd) License: BSD-3 TODOs, Design ------------- If we only care about univariate analysis, i.e. marginal if statistics returns more than one value, the we only need to store the sorted mcres not the original res. Do we want to extend to multivariate analysis? Use distribution function to keep track of MC results, ECDF, non-paramatric? Large parts are similar to a 2d array of independent multivariate random variables. Joint distribution is not used (yet). I guess this is currently only for one sided test statistics, e.g. for two-sided tests basend on t or normal distribution use the absolute value. )lrangeN) SimpleTablecneZdZdZdZdggfdZddZdZdgdfdZdd Z dd Z gdddfd Z dd Z dS) StatTestMCaclass to run Monte Carlo study on a statistical test''' TODO print(summary, for quantiles and for histogram draft in trying out script log Parameters ---------- dgp : callable Function that generates the data to be used in Monte Carlo that should return a new sample with each call statistic : callable Function that calculates the test statistic, which can return either a single statistic or a 1d array_like (tuple, list, ndarray). see also statindices in description of run Attributes ---------- many methods store intermediate results self.mcres : ndarray (nrepl, nreturns) or (nrepl, len(statindices)) Monte Carlo results stored by run Notes ----- .. Warning:: This is (currently) designed for a single call to run. If run is called a second time with different arguments, then some attributes might not be updated, and, therefore, not correspond to the same run. .. Warning:: Under Construction, do not expect stability in Api or implementation Examples -------- Define a function that defines our test statistic: def lb(x): s,p = acorr_ljungbox(x, lags=4) return np.r_[s, p] Note lb returns eight values. Define a random sample generator, for example 500 independently, normal distributed observations in a sample: def normalnoisesim(nobs=500, loc=0.0): return (loc+np.random.randn(nobs)) Create instance and run Monte Carlo. Using statindices=list(range(4)) means that only the first for values of the return of the statistic (lb) are stored in the Monte Carlo results. mc1 = StatTestMC(normalnoisesim, lb) mc1.run(5000, statindices=list(range(4))) Most of the other methods take an idx which indicates for which columns the results should be presented, e.g. print(mc1.cdf(crit, [1,2,3])[1] c"||_||_dSN)dgp statistic)selfr r s a/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/sandbox/tools/mctools.py__init__zStatTestMC.__init__ds"NcN ||_||_||_||_|j}|j}|||g|R t tj x|_ }|Btj |} |d<td|dz |D]} ||} || g|R|| <nyt |x|_ }tj ||fx|_ } fd|D|d<td|dz D]!} ||} || g|R fd|D|| <"||_ dS)arun the actual Monte Carlo and save results Parameters ---------- nrepl : int number of Monte Carlo repetitions statindices : None or list of integers determines which values of the return of the statistic functions are stored in the Monte Carlo. Default None means the entire return. If statindices is a list of integers, then it will be used as index into the return. dgpargs : tuple optional parameters for the DGP statsargs : tuple optional parameters for the statistics function Returns ------- None, all results are attached Nrc g|] }| Sr).0imcres0s r z"StatTestMC.run..s777aq 777rc g|] }| Srr)rrrets r rz"StatTestMC.run..s999SV999r) nrepl statindicesdgpargs statsargsr r lennpravelnreturnzerosrangemcres) r rrrrr statfunnreturnsr#iixrrs @@r runzStatTestMC.runhs. & "h.g3333"%bhv&6&6"7"77 x  HUOOEE!HAuQw11 3 3CM#GA2 222b  3 '*+&6&6 6DL8!#5(*;!O>OO P P P P 5!!#%xr2c|iif}|jjdkr#||jdd|f}ntd|j}tj||d}ddlm}|j } |j |f|dd|d|j |||dfi|d dS) aplot the histogram against a reference distribution Parameters ---------- idx : None or list of integers List of indices into the Monte Carlo results (columns) that should be used in the calculation distpdf : callable probability density function of reference distribution bins : {int, array_like} used unchanged for matplotlibs hist call ax : TODO: not implemented yet kwds : None or tuple of dicts extra keyword options to the calls to the matplotlib functions, first dictionary is for his, second dictionary for plot of the reference distribution Returns ------- None Nr*r+drT)r-normedrr) r#r/r0rlinspaceminmaxmatplotlib.pyplotpyplotfigurehistplot) r r8distpdfr-axkwdsr#lsppltfigs r plot_histzStatTestMC.plot_hist&s0 <r7D :?a   111S5) !GHHHJEk%))++uyy{{C88 (''''' cjll:T$::$q':::ggcllC334733333rc2tj|}|||\}}|tj|j}g} t |D]0\} } | |dd| f|dd| fg1tj|g| z} |r|dz}nd}|'dt| j ddzD}dgd |Dz} t| d d gd g| j ddz zzi|| S)asummary table for quantiles (critical values) Parameters ---------- idx : None or list of integers List of indices into the Monte Carlo results (columns) that should be used in the calculation distppf : callable probability density function of reference distribution TODO: use `crit` values instead or additional, see summary_cdf frac : array_like, float probabilities for which varnames : None, or list of strings optional list of variable names, same length as idx Returns ------- table : instance of SimpleTable use `print(table` to see results )rCNz Quantiles (critical values)zQuantiles (critical values)cg|]}d|zSzvar%drrrs r rz0StatTestMC.summary_quantiles..~sFFF! FFFrrr*z probc&g|]}dD] }|d| S))mcdist r)rrts r rz0StatTestMC.summary_quantiles..s/VVV~VV!1  VVVVr data_fmts%#6.3f%#10.4ftxt_fmttitleheaders) rrIrG atleast_2drPrLextend column_stackr"rKr)r r8distppfrCvarnamesrxquantmcqcritmmlrrTmmlarrys r summary_quantileszStatTestMC.summary_quantilesUsR.mC  ^^Cd^33 swr}U++-.. s^^ . .EAr JJAAAaC$qqqs), - - - -# ..  099EE/E  FFU5;q>13D-E-EFFFH*VVXVVVV5#. I; TUWXHX;Y0Y"Z %")+++ +rc ,tj|}g}tt|D]T}|||dd|f||gdUtj|g|z}|r|dz}nd}|'dt|jddz D}dg|z} t|ddgd gtj |jddz zzi|| S) amsummary table for cumulative density function Parameters ---------- idx : None or list of integers List of indices into the Monte Carlo results (columns) that should be used in the calculation frac : array_like, float probabilities for which crit : array_like values for which cdf is calculated varnames : None, or list of strings optional list of variable names, same length as idx Returns ------- table : instance of SimpleTable use `print(table` to see results Nrz Probabilites Probabilitiescg|]}d|zSrlrrms r rz*StatTestMC.summary_cdf..sEEE! EEErprobrsrtrurv) rrIr"rrMrUrr|rKrarray) r r8rCrr~rxrrrrys r summary_cdfzStatTestMC.summary_cdfs4.mC   s3xx A AA JJtxxQQQqS CF844Q7==?? @ @ @ @ --  "?*EE!E  EEU5;q>!3C-D-DEEEH(X%5#. I;QT H[\]H^_`H`;a0a"b %")+++ +r)NNr)NrVNN) __name__ __module__ __qualname____doc__r r(r1r?rGrUrirrrrr rr sAAF###&*26666r;;;;D !'F'F'F)>)>)>)>V0000d,4,4,4,4^4S3R3R#'t.+.+.+.+`1+1+1+1+1+1+rr__main__)statsacorr_ljungboxrXcj|tj|zSr)rrandomrandnr5)nobsdrifts r randwalksimrs'biood+++33555rcF|tj|zSr)rrr)rlocs r normalnoisesimrsBIOOD)))*rz Ljung BoxcLt|dd\}}|d|dfS)NTlags return_dfr.rr'sps r lb4rs,QQ$777!uae|rcLt|dd\}}|d|dfS)NrTrrrrs r lb1rs,QQ$777!tQqTzrcTt|dd\}}tj||fS)NrTr)rrr3rs r lbrs*QQ$777!uQT{rzResults with MC classi')rrr@)r9)rr*)r*rr)zlag 1zlag 2zlag 3r)r~rxgNё\C?g9#J?rr*r)rx)rXr)rr)(rstatsmodels.compat.pythonrnumpyrstatsmodels.iolib.tablerrrscipyrstatsmodels.stats.diagnosticrrrprintrrrmc1r(r1rGrchi2ppfrUrCrzrPrrrKdoplotr^r_rgripdfshowrrr rs2-,,,,,//////V+V+V+V+V+V+V+V+@  z;;;;;;6666++++ E- E !""" *^R ( (CGGEvvayyG))) E#--#B#B#B- C CDDD E#--   E#--   E#--   E#  GGG)<)<)@)D)D)D&6  8 8999 E#''&!   E#''&!   E#''&''' " "### * * *D 5:qe   t!4!4!6 7 7D E#//1Q%t3C/ D DEEE 5:ggg   " "=2=#6#6#8 9 9D E#//'''4#>#>#> 0  2 2333 E#''$  # )*** F '''''' a  1 )*** QJr