hMh~dZddlZddlmZmZddlmZddlmZddl m Z m Z ej ej jdz Zd!dejd ed ejfd Zd"d ejd ed ejfdZd ejded ejfdZd#d ejded ejfdZ d$dejdededededed efdZ d%dejdeded efd ZdS)&acProject: PhiK - correlation analyzer library Created: 2018/09/05 Description: Helper functions to simulate 2D datasets Authors: KPMG Advanced Analytics & Big Data team, Amstelveen, The Netherlands Redistribution and use in source and binary forms, with or without modification, are permitted according to the terms listed in the file LICENSE. N)Paralleldelayed)!get_dependent_frequency_estimates),get_chi2_using_dependent_frequency_estimates) CPP_SUPPORT_sim_2d_data_patefieldhistndatareturnc~|dkr3ttj|}|dkrt d|dd|z }|}tj||}tj||j }|S)a< Simulate a 2 dimensional dataset given a 2 dimensional pdf :param array-like hist: contingency table, which contains the observed number of occurrences in each category. This table is used as probability density function. :param int ndata: number of simulations :return: simulated data rz(ndata (or hist.sum()) has to be positiveN)npvals) intnprintsum ValueErrorravelrandom multinomialreshapeshape)r r hchcrhouthout2ds O/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/phik/simulation.py sim_2d_datars zzBGDHHJJ''(( zzCDDD aaa488:: B ((**C 9 5 4 4D Zbh ' 'F MdataseedcHtstd|j\}}tj|dtj}tj|dtj}|p$tj dt}tj ||ztj}t||||||| ||jS)a/ Simulate a two dimensional dataset with fixed row and column totals. Simulation algorithm by Patefield: W. M. Patefield, Applied Statistics 30, 91 (1981) Python implementation inspired by (C version): https://people.sc.fsu.edu/~jburkardt/c_src/asa159/asa159.html :param data: contingency table, which contains the observed number of occurrences in each category. :param seed: optional seed for the simulation, primarily for testing purposes. This table is used as probability density function. :return: simulated data z;Patefield requires a compiled extension that was not found.r)axisr)dtype)rNotImplementedErrorrrrrastypeint32rrandint NUMPY_INT_MAXemptyr rT)r!r"nrowsncolsnrowtncoltmatrixs rsim_2d_data_patefieldr23s  ! I   :LE5 GDHH!H$$ % % , ,RX 6 6E GDHH!H$$ % % , ,RX 6 6E  629$$Q 66DXeem28 4 4 4F5%tVDDD >>% ' ' ))r r$c$|dkr8tjfdtjdDS|dkr=tjfdtjdDjSt d)aC Simulate 2 dimensional data with either row or column totals fixed. :param data: contingency table, which contains the observed number of occurrences in each category. This table is used as probability density function. :param axis: fix row totals (0) or column totals (1). :return: simulated data rcTg|]$}tt|%S)listr.0ir!s r z/sim_2d_product_multinominal..es-RRRk$q'2233RRRr rc^g|])}ttj|*Sr5)r6rr,r7s rr:z/sim_2d_product_multinominal..gs/TTT!k$&)4455TTTr z%Axis should be 0 (row) or 1 (column).)rarrayrangerr,r&)r!r$s` rsim_2d_product_multinominalr>Zs qyyxRRRRU4:a==Q=QRRRSSS xTTTTuTZPQ]?S?STTTUUWW!"IJJJr multinominalmethodc|dkrt|S|dkrt|S|dkrt|dS|dkrt|dStd)a Simulate a 2 dimensional dataset given a 2 dimensional pdf Several simulation methods are provided: - multinominal: Only the total number of records is fixed. - row_product_multinominal: The row totals fixed in the sampling. - col_product_multinominal: The column totals fixed in the sampling. - hypergeometric: Both the row or column totals are fixed in the sampling. Note that this type of sampling is only available when row and column totals are integers. :param data: contingency table :param str method: sampling method. Options: [multinominal, hypergeometric, row_product_multinominal, col_product_multinominal] :return: simulated data r?hypergeometricrow_product_multinominalrcol_product_multinominalrzselected method not recognized.)rr2r>r&)r!r@s rsim_datarEls~$4   # # #$T*** - - -*4333 - - -*4333!"CDDDr log-likelihoodFvaluesnsimlambda_simulation_methodalt_hypothesisnjobsc|st|n||dkrfdt|D}n4t|fdt|D}|S)a Simulate 2D data and calculate the chi-square statistic for each simulated dataset. :param values: The contingency table. The table contains the observed number of occurrences in each category :param int nsim: number of simulations (optional, default=1000) :param str simulation_method: sampling method. Options: [multinominal, hypergeometric, row_product_multinominal, col_product_multinominal] :param str lambda_: test statistic. Available options are [pearson, log-likelihood]. :param bool alt_hypothesis: if True, simulate values directly, and not its dependent frequency estimates. :param int njobs: number of parallel jobs used for simulation. default is -1. 1 uses no parallel jobs. :returns chi2s: list of chi2 values for each simulated dataset rc2g|]}tSr5)_simulate_and_fitr8_exp_deprKrLs rr:z)sim_chi2_distribution..s']]]A"7,=wGG]]]r )n_jobsc3VK|]#}ttV$dSN)rrQrRs r z(sim_chi2_distribution..sQ'>'>,-(Bw/@'A'A'K\^e'f'f'>'>'>'>'>'>r )rr=r)rIrJrKrLrMrNchi2srTs `` @rsim_chi2_distributionrZs@NY/777SYG zz]]]]]]QVW[Q\Q\]]]&&&&'>'>'>'>'>'>16t'>'>'>>> Lr rTcHt||}t||}|S)z9split off simulate function to allow for parallellization)r@)rEr)rTrLrKsimdatasimchi2s rrQrQs+w'8999G:7GLLG Nr )rrW)r?)rFrGr?FrH)r?rG)__doc__numpyrjoblibrr statisticsrr phik.simcorerr iinfor(maxr*ndarrayrrr2r>strrEboolr6rZfloatrQr5r rris  $$$$$$$$999999DDDDDD<<<<<<<<""&* RZs0$*$* $*3$*"*$*$*$*$*NKRZKsKrzKKKK$EE"*ESEBJEEEE<JZeg"*3S,/OSadlp2CQ"2rzc!7<r