M/Ph  dZddlmZddlmZmZddlZddlZddlZ ddl m Z m Z ddl mZmZddlmZddlmZmZmZmZdd lmZdd lmZ dd lmZn.#e$r&dd l m!Z!m"Z"ed ddgZ#e#e!e"ZYnwxYwdZ$de$%dd&dDZ'e j(e'dd)e*Z+e j,ddZ-e+dddfZ.e+dddddfZ/e+dddddfZ0ddZ1ddZ2dZ3dZ4dZ5d Z6d!Z7d"Z8dd$Z9dd&Z:dd,Z;dd/ZGd3d4Z?Gd5d6Z@d7ZAd8ZBdd9ZCdd:ZDd;ZEd<ZFdd=ZGdd>ZHdd?ZId@ZJdAZKdBZLdCZMddFZNGdGdHZOdIZPdJZQdKZReSdLk r gdMZTdNeTvrBe j(gdOgjUe jVWdPdQzZXeYe4eXdReTvsdSeTvrddTlZm[Z[e[dUeTvr}e;d'dVdWdWdXddYZZ\e j(e\]dZ^eYe^eYe^dd[z de^dd\z z eYe^dd]e^d^dd_eTvrEeebdZeYdeYejeYee}e j{ebdddfd\ZZe>e jebdddfegdZe j~ee)eZeedkZe*eeeZdedPzez edPzez z z Ze eej=eedeYdeYeeYe=eeebjdZdZeedzzdz ed[edz zz z Zeedzzdz ej=eez ZeYdeteqesepesD]j\ZuZve jeheuehevz Ze jee jdegeuevgz zZeYeueveeeez eez dkke@ebjUZewe>ebdZeYdecd)D]Zue jVWdWde j(dd#gzjU\ZZejweeZe@e jeefe je jeee jeefZewZeYee jededez eedz dz d'zeej\ZZZeejx\ZZZeYdeYeeYdeYeeejxd\ZZZeYdeYedeTvre jVWdd]e j(gdzjUZde6ed\ZZe@eeZeej\ZZZeYegdgdgdgZde6ed\ZXZe>e jeXegZeYee j(gdZe j(gdZeOeeddgZgdZeYeedd%eYeedd.eedd.Ze ddgeddddPe ddedeYeeddeYdeeddeMddPddSdS)a from pystatsmodels mailinglist 20100524 Notes: - unfinished, unverified, but most parts seem to work in MonteCarlo - one example taken from lecture notes looks ok - needs cases with non-monotonic inequality for test to see difference between one-step, step-up and step-down procedures - FDR does not look really better then Bonferoni in the MC examples that I tried update: - now tested against R, stats and multtest, I have all of their methods for p-value correction - getting Hommel was impossible until I found reference for pvalue correction - now, since I have p-values correction, some of the original tests (rej/norej) implementation is not really needed anymore. I think I keep it for reference. Test procedure for Hommel in development session log - I have not updated other functions and classes in here. - multtest has some good helper function according to docs - still need to update references, the real papers - fdr with estimated true hypothesis still missing - multiple comparison procedures incomplete or missing - I will get multiple comparison for now only for independent case, which might be conservative in correlated case (?). some References: Gibbons, Jean Dickinson and Chakraborti Subhabrata, 2003, Nonparametric Statistical Inference, Fourth Edition, Marcel Dekker p.363: 10.4 THE KRUSKAL-WALLIS ONE-WAY ANOVA TEST AND MULTIPLE COMPARISONS p.367: multiple comparison for kruskal formula used in multicomp.kruskal Sheskin, David J., 2004, Handbook of Parametric and Nonparametric Statistical Procedures, 3rd ed., Chapman&Hall/CRC Test 21: The Single-Factor Between-Subjects Analysis of Variance Test 22: The Kruskal-Wallis One-Way Analysis of Variance by Ranks Test Zwillinger, Daniel and Stephen Kokoska, 2000, CRC standard probability and statistics tables and formulae, Chapman&Hall/CRC 14.9 WILCOXON RANKSUM (MANN WHITNEY) TEST S. Paul Wright, Adjusted P-Values for Simultaneous Inference, Biometrics Vol. 48, No. 4 (Dec., 1992), pp. 1005-1013, International Biometric Society Stable URL: http://www.jstor.org/stable/2532694 (p-value correction for Hommel in appendix) for multicomparison new book "multiple comparison in R" Hsu is a good reference but I do not have it. Author: Josef Pktd and example from H Raja and rewrite from Vincent Davis TODO ---- * name of function multipletests, rename to something like pvalue_correction? ) namedtuple)lziplrangeN)assert_almost_equal assert_equal)stats interpolate) SimpleTable) multipletests_ecdf fdrcorrectionfdrcorrection_twostage)utils) ValueWarning)studentized_range)qsturngpsturngrppfsf)rra 2 3 4 5 6 7 8 9 10 5 3.64 5.70 4.60 6.98 5.22 7.80 5.67 8.42 6.03 8.91 6.33 9.32 6.58 9.67 6.80 9.97 6.99 10.24 6 3.46 5.24 4.34 6.33 4.90 7.03 5.30 7.56 5.63 7.97 5.90 8.32 6.12 8.61 6.32 8.87 6.49 9.10 7 3.34 4.95 4.16 5.92 4.68 6.54 5.06 7.01 5.36 7.37 5.61 7.68 5.82 7.94 6.00 8.17 6.16 8.37 8 3.26 4.75 4.04 5.64 4.53 6.20 4.89 6.62 5.17 6.96 5.40 7.24 5.60 7.47 5.77 7.68 5.92 7.86 9 3.20 4.60 3.95 5.43 4.41 5.96 4.76 6.35 5.02 6.66 5.24 6.91 5.43 7.13 5.59 7.33 5.74 7.49 10 3.15 4.48 3.88 5.27 4.33 5.77 4.65 6.14 4.91 6.43 5.12 6.67 5.30 6.87 5.46 7.05 5.60 7.21 11 3.11 4.39 3.82 5.15 4.26 5.62 4.57 5.97 4.82 6.25 5.03 6.48 5.20 6.67 5.35 6.84 5.49 6.99 12 3.08 4.32 3.77 5.05 4.20 5.50 4.51 5.84 4.75 6.10 4.95 6.32 5.12 6.51 5.27 6.67 5.39 6.81 13 3.06 4.26 3.73 4.96 4.15 5.40 4.45 5.73 4.69 5.98 4.88 6.19 5.05 6.37 5.19 6.53 5.32 6.67 14 3.03 4.21 3.70 4.89 4.11 5.32 4.41 5.63 4.64 5.88 4.83 6.08 4.99 6.26 5.13 6.41 5.25 6.54 15 3.01 4.17 3.67 4.84 4.08 5.25 4.37 5.56 4.59 5.80 4.78 5.99 4.94 6.16 5.08 6.31 5.20 6.44 16 3.00 4.13 3.65 4.79 4.05 5.19 4.33 5.49 4.56 5.72 4.74 5.92 4.90 6.08 5.03 6.22 5.15 6.35 17 2.98 4.10 3.63 4.74 4.02 5.14 4.30 5.43 4.52 5.66 4.70 5.85 4.86 6.01 4.99 6.15 5.11 6.27 18 2.97 4.07 3.61 4.70 4.00 5.09 4.28 5.38 4.49 5.60 4.67 5.79 4.82 5.94 4.96 6.08 5.07 6.20 19 2.96 4.05 3.59 4.67 3.98 5.05 4.25 5.33 4.47 5.55 4.65 5.73 4.79 5.89 4.92 6.02 5.04 6.14 20 2.95 4.02 3.58 4.64 3.96 5.02 4.23 5.29 4.45 5.51 4.62 5.69 4.77 5.84 4.90 5.97 5.01 6.09 24 2.92 3.96 3.53 4.55 3.90 4.91 4.17 5.17 4.37 5.37 4.54 5.54 4.68 5.69 4.81 5.81 4.92 5.92 30 2.89 3.89 3.49 4.45 3.85 4.80 4.10 5.05 4.30 5.24 4.46 5.40 4.60 5.54 4.72 5.65 4.82 5.76 40 2.86 3.82 3.44 4.37 3.79 4.70 4.04 4.93 4.23 5.11 4.39 5.26 4.52 5.39 4.63 5.50 4.73 5.60 60 2.83 3.76 3.40 4.28 3.74 4.59 3.98 4.82 4.16 4.99 4.31 5.13 4.44 5.25 4.55 5.36 4.65 5.45 120 2.80 3.70 3.36 4.20 3.68 4.50 3.92 4.71 4.10 4.87 4.24 5.01 4.36 5.12 4.47 5.21 4.56 5.30 infinity 2.77 3.64 3.31 4.12 3.63 4.40 3.86 4.60 4.03 4.76 4.17 4.88 4.29 4.99 4.39 5.08 4.47 5.16 c6g|]}|S)split).0lines c/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/sandbox/stats/multicomp.py rts MMMtzz||MMMinfinity9999  皙?c|dkr-tjttdd|dz f}nB|dkr-tjttdd|dz f}nt d||S)0 return critical values for Tukey's HSD (Q) Parameters ---------- k : int in {2, ..., 10} number of tests df : int degrees of freedom of error term alpha : {0.05, 0.01} type 1 error, 1-confidence level not enough error checking for limitations r%Nr!{Gz?z1only implemented for alpha equal to 0.01 and 0.05)r interp1dcrowscv005cv001 ValueError)kdfalphaintps rget_tukeyQcritr2}sz" }}#E51Q3<88 $#E51Q3<88LMMM 488Orc4tjd|z ||S)r'r$)rr)r.r/r0s rget_tukeyQcrit2r4s"  5!R 0 00rc.tj|||S)a  return adjusted p-values for Tukey's HSD Parameters ---------- k : int in {2, ..., 10} number of tests df : int degrees of freedom of error term q : scalar, array_like; q >= 0 quantile value of Studentized Range )rr)r.r/qs rget_tukey_pvaluer7s  1b ) ))rctj|}tj|}tj|}tj|}tj|}tj|}tj|d} tj|d} tj|d} | dz| dzz| dzz} t |t |zt |zdz } | | z }tj|t |z }t |t |zt |zdz }d}||z }d}td|d}tj||z |z }tj||z |z }tj||z |z }g}t|t|t|||kr| dn| d||kr| d n| d ||kr| d n| d |S) Nr!r$?r%r03to1null3to1alt3to2null3to2alt2to1null2to1alt) npmeanstdmathpowlensqrtr2fabsprintappend)firstsecondthird firstmean secondmean thirdmeanfirststd secondstdthirdstdfirsts2seconds2thirds2 mserrornum mserrordenmserror standarderrordftotaldfgroupsdferrorqcrit qtest3to1 qtest3to2 qtest2to1 conclusions rTukeythreegenerdshIJIve}}HvIve}}Hhx##Gx 1%%Hhx##G1x!|+gk9Je**s6{{*SZZ71J#GGGrctj|}tjdt|D}||fS)Nc\g|])\}}|tjt|z*Sr)rBonesrG)rr.arrs rrzcatstack..(s1III#"'#c((+++IIIr)rBhstack enumerate)argsxlabelss rcatstackrv&s= $A YII4III J JF f9rctj|}|dddk}|dddk}tj||z|dddkzddz}|ddkrt|}nd}||fS)aYfind all up zero crossings and return the index of the highest Not used anymore >>> np.random.seed(12345) >>> x = np.random.randn(8) >>> x array([-0.20470766, 0.47894334, -0.51943872, -0.5557303 , 1.96578057, 1.39340583, 0.09290788, 0.28174615]) >>> maxzero(x) (4, array([1, 4])) no up-zero-crossing at end >>> np.random.seed(0) >>> x = np.random.randn(8) >>> x array([ 1.76405235, 0.40015721, 0.97873798, 2.2408932 , 1.86755799, -0.97727788, 0.95008842, -0.15135721]) >>> maxzero(x) (None, array([6])) Nr"rr$rBasarraynonzeromaxrtcond1cond2allzerosmaxzs rmaxzeror.s2 1 A crcFQJE abbEAIEz55=QqrrUAX677:Q>Huqyy8}} >rctj|}|dddk}|dddk}tj||z|dddkzddz}|ddkrt|}nd}||fS)aTfind all up zero crossings and return the index of the highest Not used anymore >>> np.random.seed(12345) >>> x = np.random.randn(8) >>> x array([-0.20470766, 0.47894334, -0.51943872, -0.5557303 , 1.96578057, 1.39340583, 0.09290788, 0.28174615]) >>> maxzero(x) (4, array([1, 4])) no up-zero-crossing at end >>> np.random.seed(0) >>> x = np.random.randn(8) >>> x array([ 1.76405235, 0.40015721, 0.97873798, 2.2408932 , 1.86755799, -0.97727788, 0.95008842, -0.15135721]) >>> maxzero(x) (None, array([6])) Nr"rr$rxr|s r maxzerodownrRs0 1 A crcFQJE abbEAIEz55=QqrrUAX677:Q>Huqyy8}} >rr:cjtj|t|z }||d|z z|zz }|S)zireference line for rejection in multiple tests Not used anymore from: section 3.2, page 60 r$)rBarangefloat)nr0tfrejs r rejectionlinerws8 ! U1XXA qAeG}u$ %D Krindepc tj|}tj|}||}t|}|dvr||z }n|dvr@tjdtjdt |z }||z |z}nb|dvr||d|z z|zz }nO|dvr? ' ' ' VBIc%jj)) * *u$b(/000 e^F zz|| 6**1-..  F:I: -'')) **rd2 c ||z }tjdg|z|g||z zz}g} t|D]} |t|||fz} t j| d\} } t tj| |d}ttj| |}| tj |d|tj ||dgtj |d|tj ||dgz| ztj |  ztj | d||ktj | |d|kgztj | d|||z ktj | |d||z kgztj| S)z%MonteCarlo to test fdrcorrection r)sizerrr0rr;N) rBarrayrangerandmvnr ttest_1samprabsfdrcorrection0rKrftolistsort)nreplnobsntestsntruemur0rhonfalselocsresultsrrvstttpvalresres0s rmcfdrrse^F 8RDJ"v~!66 7 7DG 5\\==WSf~6666%c1-- Eu U3GGGbfUmm5999s6E6{++RVCK-@-@AtFUF|,,bfT%&&\.B.BCDzz||$wu~~,,../uVeV}U233uUVV}U2335 5 uVeV}U6\9::uUVV}U6\9::< < = = = = 8G  rr$r!Fc|\}}d|krt|dkrntj||dz}|ddddftjd|z z|ddddftj|zz}n|dkr!tj||}n|dkr|d|dz z krt d|d|dz z kr d|dz dzz }|tj||fzd|z tj|zz}tjtj||tj |j }|rtj |}|S)acreate random draws from equi-correlated multivariate normal distribution Parameters ---------- rho : float correlation coefficient size : tuple of int size is interpreted (nobs, nvars) where each row Returns ------- rvs : ndarray nobs by nvars where each row is a independent random draw of nvars- dimensional correlated rvs rr$Nr"gz'rho has to be larger than -1./(nvars-1)g|=) rBrandomrandnrHr-roeyedotlinalgcholeskyTrzscore)rr standardizernvarsrrvs2As rrrso"KD%3ww377ioodE!G,,111SbS5zBGAcENN*S233Z"'#,,-FF qytU++ q eAg  FGG G CqM ! !uQwu}%C u && &#rve}}'< <vbioodE22BI4F4Fq4I4I4KLL"|D!! Krctjtj|t}||dk}t t |}d|dz|z |dz|z z z }|S)z: should be equivalent of scipy.stats.tiecorrect dtyper$r9)rBbincountryintrrGrf)xranks rankbincountntiesntot tiecorrections r tiecorrectrsv;rz&s;;;<)AaaacF4@@@KC [)AaaacF##C   %'[%8%88 x (((((rcF|j}|r9|dddfdz|_n|dddf|_tj|j|jx|_}|dz|jz x|_}||j|_ dS) runbasic_oldNr$rweightsr) rtrxxrBrrgroupsumr groupmeangroupmeanfilter)rrrt groupranksum grouprankmeans rrzGroupsStats.runbasic_old-s F  !fnn&&..0014DGG!fDG'){4;'P'P'PP *6);dn)LL,T[9rc|j}|rtj|dddfd\}}|dddfdz}t tj||gdj|_n|dddf|_tj|j |jx|_ }|d z|j z x|_ }||j |_dS) rNrTrr$Frrr) rtrBrrr column_stackrrrrrrr)rrrtxunixintlabranksrawrrs rrzGroupsStats.runbasic?s F  Ia!fTBBBMD'1v~~''//11A5H!"/8W2E"F"F+02222A GG!fDG'){4;'P'P'PP *6);dn)LL,T[9rc |j|jz S) groupdemean)rrrs rrzGroupsStats.groupdemeanTsw---rcf|}tj|j|dzS) groupsswithinr!r)rrBrr)rxtmps rrzGroupsStats.groupsswithinXs.!!{4;a8888rc@||jdz z S)groupvarwithinr$)rrrs rrzGroupsStats.groupvarwithin]s !!##T^A%566r)FNNF) __name__ __module__ __qualname____doc__rrrrrrrrrrrs  ))))B::::$::::*...999 77777rrc>eZdZdZ d dZdZdZdZ d dZdS) TukeyHSDResultsaOResults from Tukey HSD test, with additional plot methods Can also compute and plot additional post-hoc evaluations using this results class. Attributes ---------- reject : array of boolean, True if we reject Null for group pair meandiffs : pairwise mean differences confint : confidence interval for pairwise mean differences std_pairs : standard deviation of pairwise mean differences q_crit : critical value of studentized range statistic at given alpha halfwidths : half widths of simultaneous confidence interval pvalues : adjusted p-values from the HSD test Notes ----- halfwidths is only available after call to `plot_simultaneous`. Other attributes contain information about the data from the MultiComparison instance: data, df_total, groups, groupsunique, variance. Nc ||_||_||_||_||_||_||_||_| |_| |_ | |_ |jj |_ |jj |_ |jj |_ dSN) _multicomp_results_tableq_critr meandiffs std_pairsconfintdf_totalreject2variancepvaluesdatagroups groupsunique) r mc_object results_tabler rr r rrrrrs rrzTukeyHSDResults.__init__xs}$+  ""       O( o,  O8rc*t|jSr)strr rs r__str__zTukeyHSDResults.__str__s4&'''rc|jS)z*Summary table that can be printed )r rs rsummaryzTukeyHSDResults.summarys ""rczt|j|j|jjj|jj|_dS)zKCompute simultaneous confidence intervals for comparison of means. N)simultaneous_cir rr  groupstatsr pairindices halfwidthsrs r_simultaneous_ciz TukeyHSDResults._simultaneous_cis4*$+t} O6@ O799rrrc tj|\}}|||tddjjjg}g} fdttD} fdttD} |=| ttj ddddn|j vrtd tjj |kd d } ttD]|} j | |krt#| | | | t%| | | | z d kr|| g| | }| | | j | ddd d || | gd zd jjgdd|| | gd zd jjgddt|d kr-| ||j |ddddt| d kr-| | | j | dddd|dtj| tj| z }|d jjg|tj| |dz z tj| |dz zgdgj t4zdgz}|tjd tdz|||||nd| ||nd|S)a Plot a universal confidence interval of each group mean Visualize significant differences in a plot with one confidence interval per group instead of all pairwise confidence intervals. Parameters ---------- comparison_name : str, optional if provided, plot_intervals will color code all groups that are significantly different from the comparison_name red, and will color code insignificant groups gray. Otherwise, all intervals will just be plotted in black. ax : matplotlib axis, optional An axis handle on which to attach the plot. figsize : tuple, optional tuple for the size of the figure generated xlabel : str, optional Name to be displayed on x axis ylabel : str, optional Name to be displayed on y axis Returns ------- Figure handle to figure object containing interval plots Notes ----- Multiple comparison tests are nice, but lack a good way to be visualized. If you have, say, 6 groups, showing a graph of the means between each group will require 15 confidence intervals. Instead, we can visualize inter-group differences with a single interval for each group mean. Hochberg et al. [1] first proposed this idea and used Tukey's Q critical value to compute the interval widths. Unlike plotting the differences in the means and their respective confidence intervals, any two pairs can be compared for significance by looking for overlap. References ---------- .. [*] Hochberg, Y., and A. C. Tamhane. Multiple Comparison Procedures. Hoboken, NJ: John Wiley & Sons, 1987. Examples -------- >>> from statsmodels.examples.try_tukey_hsd import cylinders, cyl_labels >>> from statsmodels.stats.multicomp import MultiComparison >>> cardata = MultiComparison(cylinders, cyl_labels) >>> results = cardata.tukeyhsd() >>> results.plot_simultaneous() This example shows an example plot comparing significant differences in group means. Significant differences at the alpha=0.05 level can be identified by intervals that do not overlap (i.e. USA vs Japan, USA vs Germany). >>> results.plot_simultaneous(comparison_name="USA") Optionally provide one of the group names to color code the plot to highlight group means different from comparison_name. Nr!c<g|]}|j|z Srr!rrrhrs rrz5TukeyHSDResults.plot_simultaneous..)MMMaE!Htq11MMMrc<g|]}|j|zSrr&r's rrz5TukeyHSDResults.plot_simultaneous..r(roNoner.)xerrmarker linestylecolorecolorz)comparison_name not found in group names.rbr!r"z--z0.7)r.r/rz0.5z.Multiple Comparisons Between All Pairs (Tukey)g$@r$)!r create_mpl_axset_size_inchesgetattrr"r rrrrGerrorbarrr!rr-rBwhereminr{rKplotngroups set_titleset_ylimset_xlimastyperr set_yticksrset_yticklabels set_xlabel set_ylabel)rcomparison_nameaxfigsizexlabelylabelfigax1sigidxnsigidxminrangemaxrangemidxrr2ylblsrhs` @rplot_simultaneousz!TukeyHSDResults.plot_simultaneouss`B&r**S      ( ( ( 4t , , 4  ! ! # # #*4MMMMM5U;L;LMMMMMMMM5U;L;LMMM  " LLs5zz 2 2 #vS  N N N Nd&777 !LMMM8D->??B1ED3u::&& & &$Q'?:: Xd^44),Xa[(4.)I)IJLMNNMM!$$$$NN1%%%% LLtd1F #vS  N N N HHhtn%a'"do.E)F#5  2 2 2 HHhtn%a'"do.E)F#5  2 2 26{{Q U6]F"&/&"9#'-SFFF7||a U7^W"&/'":3'-U5JJJ FGGG F8  rvh// / b$/12333 bfX&&S0"&2B2BQW2LMNNNt(//44;;===D rySZZ!^44555 E""" !3vv<<< !3vv<<< r)NNNNNNNN)NNr#NN) rrrrrrrr"rQrrrrras,AEHL6:9999((((### 999HN.2xxxxxxrrc8eZdZdZd dZdZd dZdd Zdd ZdS)MultiComparisona Tests for multiple comparisons Parameters ---------- data : ndarray independent data samples groups : ndarray group labels corresponding to each data point group_order : list[str], optional the desired order for the group mean results to be reported in. If not specified, results are reported in increasing order. If group_order does not contain all labels that are in groups, then only those observations are kept that have a label in group_order. NcHt|t|kr.tdt|t|fztj|_tj|x_}|%tj|d\__n\|D]}||vrtd|ztj |_tj t|t_j dd}jD]b}tj j|kd}|t|z }tj j|kdj|<c|jjdkr`ddl}|dt"jdk} j| _j| _j| _tjdkrtd fd jD_tjtjd _jjd_tj_dS) Nz&data has %d elements and groups has %dTrz*group_order value '%s' not found in groupsirz>group_order does not contain all groups: dropping observationsr!z22 or more groups required for multiple comparisonsc<g|]}jj|kSr)rr)rr.rs rrz,MultiComparison.__init__..Ps'NNNqty!12NNNrr$)rGr-rBryrrrr groupintlabremptyrfillr8shapewarningswarnrdatali triu_indicesr rr;) rrr group_ordergrpcountnameidxrZ mask_keeps ` rrzMultiComparison.__init__(s t99F # #ETTWX^T_T_H``aa aJt$$ !z&111 f  24)FKO3Q3Q3Q /D t//# P Pf$$$H3NPPP%!# 5 5D !xD 377D    ! !$ ' ' 'E) O Oht{d233A6S!(*1Bd1J(K(KA(N %% ***  78DFFF!,4 #'#3I#>  Ii0 "k)4 t ! !A % %QRR RNNNNDttj|j|jgd|_|jj}|jj}tj|j t|}t|||d|d}tj t|j|dd|j|ddtj|dd tj|d d tj|d dddfd tj|d dddfd |dd t fd t fd t"fdt"fdt"fdt"fdtjfg}t'||jj}dd|zz|_t/|||d|d|d|d|d |d|d||d  S)at Tukey's range test to compare means of all pairs of groups Parameters ---------- alpha : float, optional Value of FWER at which to calculate HSD. Returns ------- results : TukeyHSDResults instance A results class containing relevant data and some post-hoc calculations Fr)ddofN)r/r0r rr$r!ryrzr{meandiffzp-adjlowerupperrrrz*Multiple Comparison of Means - Tukey HSD, z FWER=%4.2fr9r)rrBrrrVrrrvarrrGtukeyhsdrrrrrrrr rrrr)rr0gmeansgnobsvar_rrrs rrzMultiComparison.tukeyhsds& OTY(89 : :*)vdo1133#f++FFFvudt5NNN$t0Q;#0Q;!xA22!xA22!xAqqq!t a88!xAqqq!t a88"1v ''#+F!3"*F!3",e!4")5!1")5!1")5!1"*BH!5 !7 8 8 8$FFL4FGGG J*U23 t]CFCFCF"1vs1vs1vs1vtSVMM Mrr)Nr)r%rwr$r%) rrrrrrgrvrrrrrrSrSs +.+.+.+.\333 ((((:E@E@E@E@N/M/M/M/M/M/MrrSc|tj|dddfd\}}tj|}tj|tdddf}|dz|z }|dddf}tj||}|dz|z dz}||S)zvrankdata, equivalent to scipy.stats.rankdata just a different implementation, I have not yet compared speed NrTrrrr$)rBrrXr) rtrrr groupxsum groupxmeanrankrawrrs rrfrfs )AaaacF4888KC F##I FAaaacF333IS9,J!fnn&&((G;vw777L 3&2Q6M   rcLtj|}|}tj|}||}|}t|}tj|d\}}t dD]&} t d| dD]} t | | 'dS)zsimple ordered sequential comparison of means vals : array_like means or rankmeans for independent groups incomplete, no return, not used yet r$ryr"N)rBryrrGr]rrJ) valsr0alphafsortindr sortrevindrv1v2rrrs rcompare_orderedrs :d  D FjG ME""J YYF_VQ ' 'FB 1XXqB  A !AJJJJ rctj|}|sd|z Sd|z t|z S)aOcorrection factor for variance with unequal sample sizes this is just a harmonic mean Parameters ---------- nobs_all : array_like The number of observations for each sample srange : bool if true, then the correction is divided by the number of samples for the variance of the studentized range statistic Returns ------- correction : float Correction factor for variance. Notes ----- variance correction factor is 1/k * sum_i 1/n_i where k is the number of samples and summation is over i=0,...,k-1. If all n_i are the same, then the correction factor is 1. This needs to be multiplied by the joint variance estimate, means square error, MSE. To obtain the correction factor for the standard deviation, square root needs to be taken. r)rBryrfrG)nobs_allsranges rvarcorrection_unbalancedr(sSDz(##H 18   """8   ""3x==00rchtj||\}}|s d|z d|z zSd|z d|z zdz S)acorrection factor for variance with unequal sample sizes for all pairs this is just a harmonic mean Parameters ---------- nobs_all : array_like The number of observations for each sample srange : bool if true, then the correction is divided by 2 for the variance of the studentized range statistic Returns ------- correction : ndarray Correction factor for variance. Notes ----- variance correction factor is 1/k * sum_i 1/n_i where k is the number of samples and summation is over i=0,...,k-1. If all n_i are the same, then the correction factor is 1. This needs to be multiplies by the joint variance estimate, means square error, MSE. To obtain the correction factor for the standard deviation, square root needs to be taken. For the studentized range statistic, the resulting factor has to be divided by 2. r@rBmeshgrid)rrn1n2s rvarcorrection_pairs_unbalancedrPsKL[8 , ,FB $22 22 ##rctj|}|dz|z }|}|dz|dz|zz }||fS)areturn joint variance from samples with unequal variances and unequal sample sizes something is wrong Parameters ---------- var_all : array_like The variance for each sample nobs_all : array_like The number of observations for each sample df_all : array_like degrees of freedom for each sample Returns ------- varjoint : float joint variance. dfjoint : float joint Satterthwait's degrees of freedom Notes ----- (copy, paste not correct) variance is 1/k * sum_i 1/n_i where k is the number of samples and summation is over i=0,...,k-1. If all n_i are the same, then the correction factor is 1/n. This needs to be multiplies by the joint variance estimate, means square error, MSE. To obtain the correction factor for the standard deviation, square root needs to be taken. This is for variance of mean difference not of studentized range. rr!)rBryrf)var_allrdf_all var_over_nvarjointdfjoints rvarcorrection_unequalr|s_Pj!!G"h&J~~HkZ]V388:::G W rctj||\}}tj||\}}tj||\}}||z ||z z} | dz|||z dzz|||z dzzzz } | | fS)areturn joint variance from samples with unequal variances and unequal sample sizes for all pairs something is wrong Parameters ---------- var_all : array_like The variance for each sample nobs_all : array_like The number of observations for each sample df_all : array_like degrees of freedom for each sample Returns ------- varjoint : ndarray joint variance. dfjoint : ndarray joint Satterthwait's degrees of freedom Notes ----- (copy, paste not correct) variance is 1/k * sum_i 1/n_i where k is the number of samples and summation is over i=0,...,k-1. If all n_i are the same, then the correction factor is 1. This needs to be multiplies by the joint variance estimate, means square error, MSE. To obtain the correction factor for the standard deviation, square root needs to be taken. TODO: something looks wrong with dfjoint, is formula from SPSS r!r) rrrrrrrdf1df2rrs rvarcorrection_pairs_unequalrsR[' * *FB [8 , ,FB{66**HC"ur"u}HkSBrEA:-r"uqj0@@AG W rc Ztj|}t|}||dz }tj|dkr||z}tj||z}ntj|}tj|dkr8tj|dkr d|z|z tj||fz}nntj|dkr|t |dz}nAtj|dkrt|||\}} |dz}ntd||dddfz } tj |} tj |d\} } | | | f}| | | f}tj ||z }t|d}|t||| }t|||}tj|}||k}||z}tj ||k}tj||z ||zf}| | f||||||||f S) asimultaneous Tukey HSD check: instead of sorting, I use absolute value of pairwise differences in means. That's irrelevant for the test, but maybe reporting actual differences would be better. CHANGED: meandiffs are with sign, studentized range uses abs q_crit added for testing TODO: error in variance calculation when nobs_all is scalar, missing 1/n Nr$rTrrnot supposed to be hererr;)rBryrGrrorfrrr-rHr]rr{r4r7 atleast_1dr)mean_allrrr/r0r n_meansr var_pairsdf_sum meandiffs_ std_pairs_idx1idx2r r st_range df_total_rrcrit_intrrs rrrs\z(##H(mmG z \ wr{{aR< WW   "6":: QRWW%5%5%:%:L8+bgw6H.I.II   Q   >C "bAAkN AFFHHJA 6#;;D 6!!   D Qet^d "RBG'< = 276?? "R ' RWQZZ  ""rc*tj|}t|}||dz }tj|dkr||z}ntj|}tj|dkr8tj|dkr d|z|z tj||fz}nntj|dkr|t |dz}nAtj|dkrt|||\}}|dz}ntd||dddfz } tj |} tj |d\} } |rt| | f} t| | f} tj | | z } | | | | | ffS)zpairwise distance matrix, outsourced from tukeyhsd CHANGED: meandiffs are with sign, studentized range uses abs q_crit added for testing TODO: error in variance calculation when nobs_all is scalar, missing 1/n Nr$rTrrr)rBryrGrrfrorrr-rHr]rrr)rrrr/triurrrrr r rrrs rdistance_st_rangergsz(##H(mmG z \ wr{{aR<6":: QRWW%5%5%:%:L8+bgw6H.I.II   Q  } | tj|z} | d|| |||z ?|tj| fS)z$pvalues for simultaneous tests r) mvstdtprobNrz-df has to be specified for the t-distributionr$) .statsmodels.sandbox.distributions.multivariaterr-rBryrGrrorK) rtcorrr/dist alternativerrcc pval_globalrtilimitss rrrsJIIIII HIII Ju  E ZZF Bjj"R333K E88BGFOO# QRCE26667777  5)) ))rc>eZdZdZd dZdZdZdZdZdZ d Z dS) StepDowna0a class for step down methods This is currently for simple tree subset descend, similar to homogeneous_subsets, but checks all leave-one-out subsets instead of assuming an ordered set. Comment in SAS manual: SAS only uses interval subsets of the sorted list, which is sufficient for range tests (maybe also equal variance and balanced sample sizes are required). For F-test based critical distances, the restriction to intervals is not sufficient. This version uses a single critical value of the studentized range distribution for all comparisons, and is therefore a step-down version of Tukey HSD. The class is written so it can be subclassed, where the get_distance_matrix and get_crit are overwritten to obtain other step-down procedures such as REGW. iter_subsets can be overwritten, to get a recursion as in the many to one comparison with a control such as in Dunnet's test. A one-sided right tail test is not covered because the direction of the inequality is hard coded in check_set. Also Peritz's check of partitions is not possible, but I have not seen it mentioned in any more recent references. I have only partially read the step-down procedure for closed tests by Westfall. One change to make it more flexible, is to separate out the decision on a subset, also because the F-based tests, FREGW in SPSS, take information from all elements of a set and not just pairwise comparisons. I have not looked at the details of the F-based tests such as Sheffe yet. It looks like running an F-test on equality of means in each subset. This would also outsource how pairwise conditions are combined, any larger or max. This would also imply that the distance matrix cannot be calculated in advance for tests like the F-based ones. Ncf||_t||_||_||_||_dSr)rrGn_valsrrr/)rrrrr/s rrzStepDown.__init__s/ $ii    rcrt|j|j|}|tj|jzS)zG get_tukeyQcrit currently tukey Q, add others r;)r2rr/rBro)rr0r s rget_critzStepDown.get_crit!s3   TWEBBB ,,,,rcnt|j|j|j|j}|d|_dS)zstudentized range statisticrrN)rrrrr/distance_matrix)rdress rget_distance_matrixzStepDown.get_distance_matrix,s4!DM4>' " " G q   --g66((DMM$''''(( $$U7^^44444 M w 0 0 0Nrcdi|_|||_g|_g|_||t|jtt|jtttjfS)zmain function to run the test, could be done in __call__ instead this could have all the initialization code ) r rr rrrrrrlistsetsd)rr0s rrunz StepDown.runXsMM%((      """ fT[))***C &&''c"+.>.>)?)???rr) rrrrrrrrrrrrrrrrs  D---''' "   @@@@@rrc t|t}gg tjdkrtjfz fd  ||}fdt D}t }tt ||z }t||tt |fS)aQrecursively check all pairs of vals for minimum distance step down method as in Newman-Keuls and Ryan procedures. This is not a closed procedure since not all partitions are checked. Parameters ---------- vals : array_like values that are pairwise compared dcrit : array_like or float critical distance for rejecting, either float, or 2-dimensional array with distances on the upper triangle. Returns ------- rejs : list of pairs list of pair-indices with (strictly) larger than critical difference nrejs : list of pairs list of pair-indices with smaller than critical difference lli : list of tuples list of subsets with smaller than critical difference res : tree result of all comparisons (for checking) this follows description in SPSS notes on Post-Hoc Tests Because of the recursive structure, some comparisons are made several times, but only unique pairs or sets are returned. Examples -------- >>> m = [0, 2, 2.5, 3, 6, 8, 9, 9.5,10 ] >>> rej, nrej, ssli, res = homogeneous_subsets(m, 2) >>> set_partition(ssli) ([(5, 6, 7, 8), (1, 2, 3), (4,)], [0]) >>> [np.array(m)[list(pp)] for pp in set_partition(ssli)[0]] [array([ 8. , 9. , 9.5, 10. ]), array([ 2. , 2.5, 3. ]), array([ 6.])] r$cz|d|d}}|d|dz ||fkrj|d|df|dd|dd|dd|dd|d|dfgSt||S)zwrecursive function for constructing homogeneous subset registers rejected and subsetli in outer scope rr"Nr$)rKr )rindices_rrrdcritrsubsets subsetslis rrz$homogeneous_subsets..subsetss  Xb\1 8d1g ac * * OOXa[(2,7 8 8 8GD"Ix}55GDHhqrrl33a[(2,/1 1   U8__ - - -OrcFg|]}tdz |dD]}||fS)r$r")r)rrrrnvalss rrz'homogeneous_subsets..s:JJJ1eE!GAb6I6IJJ!AJJJJr)rGrrBrrorrr) rrrr all_pairsrejs not_rejectedr"rrr s ` @@@@rhomogeneous_subsetsr&msV IIEe}}HHI wu~~bguen---         '$ ! !CJJJJe JJJI x==DI-..L ::|T#i..%9%93 >>rcvg}ttt|tdddD]S}t|t fd|Ds||Ttd|Dd|Dz }||fS)aextract a partition from a list of tuples this should be correctly called select largest disjoint sets. Begun and Gabriel 1981 do not seem to be bothered by sets of accepted hypothesis with joint elements, e.g. maximal_accepted_sets = { {1,2,3}, {2,3,4} } This creates a set partition from a list of sets given as tuples. It tries to find the partition with the largest sets. That is, sets are included after being sorted by length. If the list does not include the singletons, then it will be only a partial partition. Missing items are singletons (I think). Examples -------- >>> li [(5, 6, 7, 8), (1, 2, 3), (4, 5), (0, 1)] >>> set_partition(li) ([(5, 6, 7, 8), (1, 2, 3)], [0, 4]) keyNr"c3vK|]3}tt|V4dSr)r intersection)rrs_s r z set_partition..s=>>A3r77''A//>>>>>>rch|] }|D]}| Srrrllrs r z set_partition..s%111"b11A1111rch|] }|D]}| Srrr/s rr1z set_partition..s%333B331a3333r)sortedrrrGrrrK)sslipartsmissingr,s @r set_partitionr8s. D DTOO - - -ddd 3 VV[[]]>>>>>>>>>  KKNNN1111133$333455G =rcg}ttt|ddddD]2tfd|Ds|3|S)aKremove sets that are subsets of another set from a list of tuples Parameters ---------- ssli : list of tuples each tuple is considered as a set Returns ------- part : list of tuples new list with subset tuples removed, it is sorted by set-length of tuples. The list contains original tuples, duplicate elements are not removed. Examples -------- >>> set_remove_subs([(0, 1), (1, 2), (1, 2, 3), (0,)]) [(1, 2, 3), (0, 1)] >>> set_remove_subs([(0, 1), (1, 2), (1,1, 1, 2, 3), (0,)]) [(1, 1, 1, 2, 3), (0, 1)] c:tt|Sr)rGr)rts rz!set_remove_subs..s3s1vv;;rr(Nr"c3vK|]3}tt|V4dSr)rissubset)rrr6s rr-z"set_remove_subs..s;99q3q66??3q66**999999r)r3rrrrK)r4r5r6s @rset_remove_subsr>s. D DTOO)>)> ? ? ?" E9999D99999  KKNNN Kr__main__)tukey tukeycritfdrfdrmcrwr multicompdevr+r@)rrr$r9rBrw)example_fdr_bonferronirCig?g333333?)rrrrrr0rg@g@ryrg?)ir)rowvarrArr;gffffff@r(gzG@rDgQ@g(\@g@gzG@gQ@g ףp= @gףp= @g(\@gQ@g333333@g= ףp=@g@gGz@g\(\@g)\(@gffffff@g{Gz@gp= ף@cRg|]$}ttdddf|kdf%S)Nr$r)rrr.s rrr-s0222!q111Q31~222rrcNg|]"}ttdddf|k#S)Nr$)rrrKs rrr/s-:::!F1QQQqS619%:::rc,g|]}t|Sr)rG)rxvals rrr0s444#d))444rc6g|]}|Sr)rCritems rrr1s 666TTYY[[666rc6g|]}|Sr)rfrPs rrr2s 4444DHHJJ444rgIRk?g?z sorted rank differencesz kruskal for all pairs)use_continuityg?g?Trrrrz# groupmeanfilter and grouprankmeansz! tiecorrection for data and ranksiriz pairs of mean rank differencesrjz% examples for kruskal multicomparisonr3hs)rlast<)rrr:r:)rr rrX)rrrrr)ryrrrr)g{G@gQ @g9 #@gR1@)rrrrgMbP?)g-C6?g-C6:?gŏ1w-!_?g~jt?g0*?gPkw?g2%䃞?g"u?g9m4?g#~j?gfc]F?g:M?gZӼ?g}?5^I?rr)r0itergu!Va?gQI?fdr_gbsgx&1 @r)r:)r%r)rrrrr:r%r)rFr)Nr%Nr)NF)Nrr)r collectionsrstatsmodels.compat.pythonrrrrEnumpyrB numpy.testingrrscipyrr statsmodels.iolib.tabler statsmodels.stats.multitestr r rr rrstatsmodels.graphicsrstatsmodels.tools.sm_exceptionsr scipy.statsr ImportErrorstatsmodels.stats.libqsturngrrstudentized_range_tupler_replacerrrr?rcrccolsr*r+r,r2r4r7rdrlrvrrrrrrrrrrSrfrrrrrrrrrrrrrrr&r8r>rexamplesrrrrtrJ ex_multicomprFmcresrCmcmeansrvsmvncorrcoefrrrxlirxrankslixnobsrpsumranksrlrisfrmrsr]rrrdiffidxrkrrrrv mannwhitneyumwumwupvalrrrrrrrrrrgsrrrrgs2rrrrGrrfrrYrorrqrrsrHrt multicompgsrx1x2skwr_rromc2newskwr ttest_indtablettresttarrtttablemwresmwarrmw tablemwhsxrvsxrvsgr multicomprrugs4rrrrrres_tstrrrrsu>>~#"""""22222222 ;;;;;;;;$$$$$$$$//////}}}}}}}}}}}}&&&&&&888888I-------III========(j)5>!AAAaC&3I3I2NNN [T * * * 4555 b !!! mF#$$$" !AAAaC&>>> gk/"/1QQQqS67*;<>>LGAJNN$$%%% E d > > >??? E)]]5YGGGHHHM%B]s A(A>=A>