hMh dZddlmZddlZddlmZdejdejfdZ ddejd e de fd Z dd ejd ejde de fdZ deeejfde fdZd ejdefdZd ejdefdZdde de de fdZdS)aProject: PhiK - correlation coefficient package Created: 2018/09/05 Description: Statistics helper functions, for the calculation of phik and significance of a contingency table. Authors: KPMG Advanced Analytics & Big Data team, Amstelveen, The Netherlands Redistribution and use in source and binary forms, with or without modification, are permitted according to the terms listed in the file LICENSE. )UnionN)statsvalsreturnc@tj|S)a0 Calculation of dependent expected frequencies. Calculation is based on the marginal sums of the table, i.e. dependent frequency estimates. :param vals: The contingency table. The table contains the observed number of occurrences in each category :returns exp: expected frequencies )r contingency expected_freq)rs O/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/phik/statistics.py!get_dependent_frequency_estimatesr s   * *4 0 00log-likelihoodlambda_c|dd}|tj|dkd}|jtj|jdkdj}tj||\}}}}|S)ag Chi-square test of independence of variables in a contingency table. The expected frequencies are based on the marginal sums of the table, i.e. dependent frequency estimates. :param vals: The contingency table. The table contains the observed number of occurrences in each category :returns test_statistic: the test statistic value Nr)axis)r)npallTrchi2_contingency)rrvaluestest_statistic_s r ,get_chi2_using_dependent_frequency_estimatesr$s!!!WFRVFaKa0000 1F Xrvfh!m!4444 5 7F$4VWMMMNAq! r Tobservedexpected normalizectj|}tj|dkrtd|jdkrtd|t |}tj|}|r,|tj|tj|z z}tj|tj |z dz|tj ||dk}tj|S)aCalculate pearson chi square between observed and expected 2d contingency matrix :param observed: The observed contingency table. The table contains the observed number of occurrences in each cell. :param expected: The expected contingency table. The table contains the expected number of occurrences in each cell. :param bool normalize: normalize expected frequencies, default is True. :return: the pearson chi2 value rz.All values in `observed` must be non-negative.zNo data; `observed` has size 0.N)outwhere) rasarrayany ValueErrorsizer sumdivideastypefloat64 zeros_like)rrrtermss r get_pearson_chi_squarer+>sz(##H vhlKIJJJ}:;;;4X>>z(##HDrvh//"&2B2BBC I  $ $x /A5 M( # #!m    E 6%==r chi2valuesc*tj|S)a! Estimation of the effective number of degrees of freedom. A good approximation of endof is the average value. Alternatively a fit to the chi2 distribution can be make. Both values are returned. :param list chi2values: list of chi2 values :returns: endof0, endof )rmean)r,s r estimate_ndofr/_s 7:  r ctj|}|jt j|jz |jzdz |dkz }|dkrd}|S)a Simple estimation of the effective number of degrees of freedom. This equals the nominal calculation for ndof minus the number of empty bins in the expected contingency table. :param observed: numpy array of observed cell counts :returns: endof rr)rrr r$rr%shapendim)rrendofs r estimate_simple_ndofr4ms{ ..x88H &  ! -    q=        qyy Lr cZ|jtj|jz |jzdz S)a Simple estimation of the effective number of degrees of freedom. This equals the nominal calculation for ndof minus the number of empty bins in the expected contingency table. :param observed: numpy array of observed cell counts :returns: theoretical ndof r)r$rr%r1r2)rs r theoretical_ndofr6s) =26(.11 1HM AA EEr Flogp flip_signc|tj kr|s tjn tj Stj|}|dkrTdtjdtjzzd|zz }tj|tj|z }n t j| }|r|dz}|S)a) Convert logarithm of p-value into one-sided Z-value :param float logp: logarithm of p-value, should not be greater than 0 :param bool flip_sign: flip sign of Z-value, e.g. use for input log(1-p). Default is false. :returns: statistical significance Z-value :rtype: float rgrg@g) rinfexplogpisqrtrnormppf)r7r8p_valueuz_values r z_from_logprDs w&3rvvRVG3fTllG!|| 26!be)$$ $sTz 1'!bfQii-((:>>'***4 Nr )r )NT)F)__doc__typingrnumpyrscipyrndarrayr strfloatrboolr+listr/intr4r6rDr r rPs 1BJ 12: 1 1 1 1&6 *" 6JNj$&JBF B eD"*$45 %    2:#4 Frz Fc F F F Fer