M/PhbO dZddlZddlmZddlmZddlZddlm Z GddZ d d Z d!d Z Gd d Z GddZ dZdZd"dZdZedkrejgdZee eee edee ejdddeee dddeeejdejdddeeejddddSdS)#arunstest formulas for mean and var of runs taken from SAS manual NPAR tests, also idea for runstest_1samp and runstest_2samp Description in NIST handbook and dataplot does not explain their expected values, or variance Note: There are (at least) two definitions of runs used in literature. The classical definition which is also used here, is that runs are sequences of identical observations separated by observations with different realizations. The second definition allows for overlapping runs, or runs where counting a run is also started after a run of a fixed length of the same kind. TODO * add one-sided tests where possible or where it makes sense N)stats)comb) array_likec eZdZdZdZddZdS)Runsa=class for runs in a binary sequence Parameters ---------- x : array_like, 1d data array, Notes ----- This was written as a more general class for runs. This has some redundant calculations when only the runs_test is used. TODO: make it lazy The runs test could be generalized to more than 1d if there is a use case for it. This should be extended once I figure out what the distribution of runs of any length k is. The exact distribution for the runs test is also available but not yet verified. c8tj||_tjtjtjtj g|tjgfdx|_}tj|x|_}||ddx|_ }||dk|_ ||dk|_ tj ||_ t|j|_|dk|_dS)Nr)npasarrayxnonzerodiffr_infrunstartruns runs_signruns_posruns_negbincount runs_freqslenn_runssumn_pos)selfr rrrs ^/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/sandbox/stats/runs.py__init__z Runs.__init__8sA#%:bgbebfWIq26(floatastypeintrr6)r cutoffr- xindicators rrunstest_1samprEjs4 1cA  8  1vv+%%c**J    % % % < <>H% j!!++z+BBB 658 # 658 #:b>>H% j!!++z+BBBiB  Bx}r2hsm++H% J))Z)@@@r c0eZdZdZdZdZdZdZdZdS) TotalRunsProba_class for the probability distribution of total runs This is the exact probability distribution for the (Wald-Wolfowitz) runs test. The random variable is the total number of runs if the sample has (n0, n1) observations of groups 0 and 1. Notes ----- Written as a class so I can store temporary calculations, but I do not think it matters much. Formulas taken from SAS manual for one-sided significance level. Could be converted to a full univariate distribution, subclassing scipy.stats.distributions. *Status* Not verified yet except for mean. cd||_||_||zx|_}t|||_dSN)n0n1r.rcomball)rrfrgr.s rrzTotalRunsProb.__init__s3"WAr{{ r c|j|j}}t|dz |dzdz }t|dz |dzdz }||zdz|jz S)Nr r&r"rfrgrrh)rrrfrgtmp0tmp1s rruns_prob_evenzTotalRunsProb.runs_prob_evens[$'BBqD!Q$q&!!BqD!Q$q&!!d{R$,..r c|j|j}}|dzdz}t|dz |dz }t|dz |dz }t|dz |dz }t|dz |dz }||z||zz|jz S)Nr r&rj) rrkrfrgkrlrmtmp3tmp4s r runs_prob_oddzTotalRunsProb.runs_prob_odd"s$'B qS1HBqD!A#BqD!A#BqD!A#BqD!A#t dTk)dl::r ctj|}tj|ddk}||}||}tj|j}||||<||||<|S)Nr&r)r r modrHshapersrn)rrkr_isoddr_oddr_evenruns_pdfs rpdfzTotalRunsProb.pdf+s JqMM&A,,"' G88AG$$ ..u55!0088'r ctjd|dz}||ddd}|||dddz }|S)Nr&r )r rJrnrrs)rrkrcdfvals rcdfzTotalRunsProb.cdf6ss Yq1  $$R!W--1133$$$R1X..22444 r N) r7r8r9r:rrnrsr{r~r;r rrcrcsi0### /// ;;;r rcceZdZdZdZdZdS)RunsProbadistribution of success runs of length k or more (classical definition) The underlying process is assumed to be a sequence of Bernoulli trials of a given length n. not sure yet, how to interpret or use the distribution for runs of length k or more. Musseli also has longest success run, and waiting time distribution negative binomial of order k and geometric of order k need to compare with Godpole need a MonteCarlo function to do some quick tests before doing more cJd|z }tj||dz|dzzdzdddf}d||z zt||z|||zzz||dz zzt|||zz |dz |t|||zz |zzz}|dS)aGdistribution of success runs of length k or more Parameters ---------- x : float count of runs of length n k : int length of runs n : int total number of observations or trials p : float probability of success in each Bernoulli trial Returns ------- pdf : float probability that x runs of length of k are observed Notes ----- not yet vectorized References ---------- Muselli 1996, theorem 3 r Nr r)r rJrr)rr rpr.pqmtermss rr{z RunsProb.pdfRs8 aC Ia!A#1a ( (4 0qs d1ajj(1qs83a!A#h>AaCQ''!d1qs7A.>.>*>>@yy||r cdSrer;)rr rpr.rs rpdf_nbzRunsProb.pdf_nbts r N)r7r8r9r:r{rr;r rrr=s=(   D     r rc tjtj}fd|D}tj tj fd|D}tjd|D}||z }|} k}||z } tj||f} tj|dz|z | z|dz|z |zf} | dkrtdtj | | d| | fS) achisquare test for equality of median/location This tests whether all groups have the same fraction of observations above the median. Parameters ---------- x : array_like data values stacked for all groups groups : array_like group labels or indicator Returns ------- stat : float test statistic pvalue : float pvalue from the chisquare distribution others ???? currently some test output, table and expected c(g|]}|kSr;r;).0grouprTr s r z'median_test_ksample..s" / / /1VU]  / / /r c@g|]}|kSr;)r)rxgxmedians rrz'median_test_ksample..s)AAArrG|0022AAAr c,g|]}t|Sr;)r)rrs rrz'median_test_ksample..s---2s2ww---r r#zWarning: There are cells with less than 5 expectedobservations. The chisquare distribution might not be a goodapproximation for the true distribution.r )ddof) r r rKr>arrayrvstackanyrPr chisquareravel) r rTrUxli counts_largercountscounts_smallernobsn_larger n_smallertableexpectedrs `` @rmedian_test_ksamplerss. 1 A If  E / / / / / / / /CillGHAAAASAAABBM X----- . .Fm+N ::<#?#??AF 5:==1-- --r ctjdttj|}|M|jd|jdkr1|jddkrt d|d|d}}n2tj||kd}tj||kd}|rRtj||}tj |||zd dz}tj|d}nWt|}tj ||z |z dzd ||zzz }d} tj|| }||fS) a#McNemar test Parameters ---------- x, y : array_like two paired data samples. If y is None, then x can be a 2 by 2 contingency table. x and y can have more than one dimension, then the results are calculated under the assumption that axis zero contains the observation for the samples. exact : bool If exact is true, then the binomial distribution will be used. If exact is false, then the chisquare distribution will be used, which is the approximation to the distribution of the test statistic for large sample sizes. correction : bool If true, then a continuity correction is used for the chisquare distribution (if exact is false.) Returns ------- stat : float or int, array The test statistic is the chisquare statistic if exact is false. If the exact binomial distribution is used, then this contains the min(n1, n2), where n1, n2 are cases that are zero in one sample but one in the other sample. pvalue : float or array p-value of the null hypothesis of equal effects. Notes ----- This is a special case of Cochran's Q test. The results when the chisquare distribution is used are identical, except for continuity correction. +Deprecated, use stats.TableSymmetry insteadNrr r&ztable needs to be 2 by 2)r r)rr r%r#)rrrr r rvrMrminimumrbinomr~rBr,rr+) r rSexactr-rgn2statr5corrdfs rmcnemarrs@J M?OOO 1 AyQWQZ171:-- 71:??788 84!D'BVAE1   VAE1   'z"b!!{tR"Wc22Q6z$"":rBw$&*bBGn= z}}T2&& :r ctjdttj|}|j\}}||krt dtj|d}|j|}||}||z dz||zdzz }||dz zdz }tj ||}|||fS)aDTest for symmetry of a (k, k) square contingency table This is an extension of the McNemar test to test the Null hypothesis that the contingency table is symmetric around the main diagonal, that is n_{i, j} = n_{j, i} for all i, j Parameters ---------- table : array_like, 2d, (k, k) a square contingency table that contains the count for k categories in rows and columns. Returns ------- statistic : float chisquare test statistic p-value : float p-value of the test statistic based on chisquare distribution df : int degrees of freedom of the chisquare distribution Notes ----- Implementation is based on the SAS documentation, R includes it in `mcnemar.test` if the table is not 2 by 2. The pvalue is based on the chisquare distribution which requires that the sample size is not very small to be a good approximation of the true distribution. For 2x2 contingency tables exact distribution can be obtained with `mcnemar` See Also -------- mcnemar rztable needs to be squarer r&g#B ;r") rrrr r rvrM triu_indicesTrrrr+) rrpk2upp_idxtriltriurrr5s rsymmetry_bowkerr8sP M?OOO Ju  E KEArBww3444oa##G 77 D >D D[1 t e 3 4 9 9 ; ;D acRB :==r " "D r>r __main__)r r r rrr rr rr r r rr rr )rCr )rT dr&)r)r<T)NNT)NTT)r:numpyr scipyr scipy.specialrrstatsmodels.tools.validationrrrErarcrrrrrr7rx1rPr6rJr~randomrandnrandintr;r rrs*333333LLLLLLLL\"="="="=HlAlAlAlA^========@8 8 8 8 8 8 8 8 v "-U-U-Ud<.<.<.|<<<<~999x z BBB C CB E$$r((     E..F + + +,,, E..2a++B 7 7 7888 E--!    $ $%%% E  biooc22BI4E4Ea#4N4N O OPPP E**RY&&q733 4 455555r