M/PhezdZddlZddlmZddlmZmZmZm Z m Z gdZ GddeZ Gd d eZ dS) aZ Multivariate Conditional and Unconditional Kernel Density Estimation with Mixed Data Types. References ---------- [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) [2] Racine, Jeff. "Nonparametric Econometrics: A Primer," Foundation and Trends in Econometrics: Vol 3: No 1, pp1-88. (2008) http://dx.doi.org/10.1561/0800000009 [3] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) [4] Racine, J. Li, Q. "Kernel Estimation of Multivariate Conditional Distributions Annals of Economics and Finance 5, 211-235 (2004) [5] Liu, R., Yang, L. "Kernel estimation of multivariate cumulative distribution function." Journal of Nonparametric Statistics (2008) [6] Li, R., Ju, G. "Nonparametric Estimation of Multivariate CDF with Categorical and Continuous Data." Working Paper [7] Li, Q., Racine, J. "Cross-validated local linear nonparametric regression" Statistica Sinica 14(2004), pp. 485-512 [8] Racine, J.: "Consistent Significance Testing for Nonparametric Regression" Journal of Business & Economics Statistics [9] Racine, J., Hart, J., Li, Q., "Testing the Significance of Categorical Predictor Variables in Nonparametric Regression Models", 2006, Econometric Reviews 25, 523-544 N)kernels) GenericKDEEstimatorSettingsgpke LeaveOneOut _adjust_shape)KDEMultivariateKDEMultivariateConditionalrcHeZdZdZd dZdZdfdZd dZd dZd Z d Z dS) r a Multivariate kernel density estimator. This density estimator can handle univariate as well as multivariate data, including mixed continuous / ordered discrete / unordered discrete data. It also provides cross-validated bandwidth selection methods (least squares, maximum likelihood). Parameters ---------- data : list of ndarrays or 2-D ndarray The training data for the Kernel Density Estimation, used to determine the bandwidth(s). If a 2-D array, should be of shape (num_observations, num_variables). If a list, each list element is a separate observation. var_type : str The type of the variables: - c : continuous - u : unordered (discrete) - o : ordered (discrete) The string should contain a type specifier for each variable, so for example ``var_type='ccuo'``. bw : array_like or str, optional If an array, it is a fixed user-specified bandwidth. If a string, should be one of: - normal_reference: normal reference rule of thumb (default) - cv_ml: cross validation maximum likelihood - cv_ls: cross validation least squares defaults : EstimatorSettings instance, optional The default values for (efficient) bandwidth estimation. Attributes ---------- bw : array_like The bandwidth parameters. See Also -------- KDEMultivariateConditional Examples -------- >>> import statsmodels.api as sm >>> nobs = 300 >>> np.random.seed(1234) # Seed random generator >>> c1 = np.random.normal(size=(nobs,1)) >>> c2 = np.random.normal(2, 1, size=(nobs,1)) Estimate a bivariate distribution and display the bandwidth found: >>> dens_u = sm.nonparametric.KDEMultivariate(data=[c1,c2], ... var_type='cc', bw='normal_reference') >>> dens_u.bw array([ 0.39967419, 0.38423292]) Nc||_t|j|_t||j|_||_t j|j\|_|_|j|jkrtd|tn|}| ||j s| ||_dS|||_dS)NzGThe number of observations must be larger than the number of variables.)var_typelenk_varsr data data_typenpshapenobs ValueErrorr _set_defaults efficient _compute_bwbw_compute_efficient)selfrrrdefaultss h/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/nonparametric/kernel_density.py__init__zKDEMultivariate.__init__es  $-(( !$ 44 !!#$)!4!4 4; 9 # #=>> >*2*:$&&& 8$$$~ 2&&r**DGGG--b11DGGGcd}|dt|jzdzz }|dt|jzdzz }|d|jzdzz }|d|jzdzz }|S) Provide something sane to print.z KDE instance zNumber of variables: k_vars =  zNumber of samples: nobs = zVariable types: BW selection method: )strrrr _bw_methodrrprs r__repr__zKDEMultivariate.__repr__usv /#dk2B2BBTII -DI>EE &6== &84?? r c|SNxs rzKDEMultivariate.~r c t|j}d}t|D]<\}}t|| |j|ddf |j}|||z }=| S)aX Returns the leave-one-out likelihood function. The leave-one-out likelihood function for the unconditional KDE. Parameters ---------- bw : array_like The value for the bandwidth parameter(s). func : callable, optional Function to transform the likelihood values (before summing); for the log likelihood, use ``func=np.log``. Default is ``f(x) = x``. Notes ----- The leave-one-out kernel estimator of :math:`f_{-i}` is: .. math:: f_{-i}(X_{i})=\frac{1}{(n-1)h} \sum_{j=1,j\neq i}K_{h}(X_{i},X_{j}) where :math:`K_{h}` represents the generalized product kernel estimator: .. math:: K_{h}(X_{i},X_{j}) = \prod_{s=1}^{q}h_{s}^{-1}k\left(\frac{X_{is}-X_{js}}{h_{s}}\right) rNr data_predictr)rr enumeraterr)rrfuncLOOLiX_not_if_is rloo_likelihoodzKDEMultivariate.loo_likelihood~s}6$)$$ #C..  JAwr !QQQ$7G $ ///C cNAAr r c R||j}nt||j}g}tt j|dD]I}|t|j|j||ddf|j |j z Jt j |}|S)aT Evaluate the probability density function. Parameters ---------- data_predict : array_like, optional Points to evaluate at. If unspecified, the training data is used. Returns ------- pdf_est : array_like Probability density function evaluated at `data_predict`. Notes ----- The probability density is given by the generalized product kernel estimator: .. math:: K_{h}(X_{i},X_{j}) = \prod_{s=1}^{q}h_{s}^{-1}k\left(\frac{X_{is}-X_{js}}{h_{s}}\right) Nrr2 rr rrangerrappendrrrrsqueeze)rr3pdf_estr8s rpdfzKDEMultivariate.pdfs,  9LL(t{CCLrx --a011 E EA NN4di-9!QQQ$-?)-888:>)D E E E E*W%%r c X||j}nt||j}g}tt j|dD]L}|t|j|j||ddf|j ddd|j z Mt j |}|S)a Evaluate the cumulative distribution function. Parameters ---------- data_predict : array_like, optional Points to evaluate at. If unspecified, the training data is used. Returns ------- cdf_est : array_like The estimate of the cdf. Notes ----- See https://en.wikipedia.org/wiki/Cumulative_distribution_function For more details on the estimation see Ref. [5] in module docstring. The multivariate CDF for mixed data (continuous and ordered/unordered discrete) is estimated by: .. math:: F(x^{c},x^{d})=n^{-1}\sum_{i=1}^{n}\left[G(\frac{x^{c}-X_{i}}{h})\sum_{u\leq x^{d}}L(X_{i}^{d},x_{i}^{d}, \lambda)\right] where G() is the product kernel CDF estimator for the continuous and L() for the discrete variables. Used bandwidth is ``self.bw``. Nr gaussian_cdfaitchisonaitken_cdf wangryzin_cdf)rr3rckertypeukertypeokertyper=)rr3cdf_estr8s rcdfzKDEMultivariate.cdfs>  9LL(t{CCLrx --a011 G GA NN4di-9!QQQ$-?)-)7)>)8 ::: =AI F G G G G*W%%r c d}ttjtjtj}|j}|j }|j}tj d|D}|| }tj |j } t|D]}} t|D]7\} } || || |dd| f|| | f| dd| f<8| d|z } | d}||z }~ttjtjtj}t'|j}d}tj |j ddz |j df} t|D]\} }t|D]8\} } || || |dd| f || | f| dd| f<9| d|z } || dz }||dzz d|z||dz zz z S)a Returns the Integrated Mean Square Error for the unconditional KDE. Parameters ---------- bw : array_like The bandwidth parameter(s). Returns ------- CV : float The cross-validation objective function. Notes ----- See p. 27 in [1]_ for details on how to handle the multivariate estimation with mixed data types see p.6 in [2]_. The formula for the cross-validation objective function is: .. math:: CV=\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{j=1}^{N} \bar{K}_{h}(X_{i},X_{j})-\frac{2}{n(n-1)}\sum_{i=1}^{n} \sum_{j=1,j\neq i}^{N}K_{h}(X_{i},X_{j}) Where :math:`\bar{K}_{h}` is the multivariate product convolution kernel (consult [2]_ for mixed data types). References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) r)coucg|]}|dk S)rMr,).0rMs r z(KDEMultivariate.imse..0s777AH777r Nraxis)dictrgaussian_convolutionwang_ryzin_convolutionaitchison_aitken_convolutionrrrrarrayprodemptyrr>r4sumgaussian wang_ryzinaitchison_aitkenr)rrFkertypesrrrix_cont_bw_cont_productKvalr8iivtypedens k_bar_sumr6r7r9s rimsezKDEMultivariate.imsespf '6!8!>@@@y z=(77h77788g;++--x ##t  A&x00 ; ; E-huobf.2111b5k.21b5k;;QQQU 99!9$$'77Da((I NAA'*!,!2444$)$$ xAq$*Q-899#C.. " "JAw&x00 ; ; E-huobf/6qqq"u~o.21b5k;;QQQU 99!9$$'77D q!! !AAD!G a!ettax'899:r cd}|jf}||fS)@Helper method to be able to pass needed vars to _compute_subset.r )rr class_type class_varss r_get_class_vars_typez$KDEMultivariate._get_class_vars_typeNs& m& :%%r NNr+ __name__ __module__ __qualname____doc__rr)r;rBrKrjrpr,r rr r )s::v2222 '2k""""H""""H....`V;V;V;p&&&&&r r cJeZdZdZ d dZdZdfdZd dZd dZd Z d Z dS) r a Conditional multivariate kernel density estimator. Calculates ``P(Y_1,Y_2,...Y_n | X_1,X_2...X_m) = P(X_1, X_2,...X_n, Y_1, Y_2,..., Y_m)/P(X_1, X_2,..., X_m)``. The conditional density is by definition the ratio of the two densities, see [1]_. Parameters ---------- endog : list of ndarrays or 2-D ndarray The training data for the dependent variables, used to determine the bandwidth(s). If a 2-D array, should be of shape (num_observations, num_variables). If a list, each list element is a separate observation. exog : list of ndarrays or 2-D ndarray The training data for the independent variable; same shape as `endog`. dep_type : str The type of the dependent variables: c : Continuous u : Unordered (Discrete) o : Ordered (Discrete) The string should contain a type specifier for each variable, so for example ``dep_type='ccuo'``. indep_type : str The type of the independent variables; specified like `dep_type`. bw : array_like or str, optional If an array, it is a fixed user-specified bandwidth. If a string, should be one of: - normal_reference: normal reference rule of thumb (default) - cv_ml: cross validation maximum likelihood - cv_ls: cross validation least squares defaults : Instance of class EstimatorSettings The default values for the efficient bandwidth estimation Attributes ---------- bw : array_like The bandwidth parameters See Also -------- KDEMultivariate References ---------- .. [1] https://en.wikipedia.org/wiki/Conditional_probability_distribution Examples -------- >>> import statsmodels.api as sm >>> nobs = 300 >>> c1 = np.random.normal(size=(nobs,1)) >>> c2 = np.random.normal(2,1,size=(nobs,1)) >>> dens_c = sm.nonparametric.KDEMultivariateConditional(endog=[c1], ... exog=[c2], dep_type='c', indep_type='c', bw='normal_reference') >>> dens_c.bw # show computed bandwidth array([ 0.41223484, 0.40976931]) Nc||_||_||z|_t|j|_t|j|_t ||j|_t ||j|_tj |j\|_ |_tj |j|jf|_ tj |j d|_|tn|}|||js|||_dS|||_dS)Nr)dep_type indep_typerrk_depk_indepr endogexogrrr column_stackrrrrrrrr)rr}r~ryrzrrs rrz#KDEMultivariateConditional.__init__s  $!J.'' 4?++ "5$*55 !$ 55 " 4 4 4:OTZ$;<< hty))!, *2*:$&&& 8$$$~ 2&&r**DGGG--b11DGGGr cd}|dt|jzdzz }|dt|jzdzz }|dt|jzdzz }|d|jzdzz }|d|jzdzz }|d|jzdzz }|S) r"z$KDEMultivariateConditional instance z+Number of independent variables: k_indep = r#z'Number of dependent variables: k_dep = zNumber of observations: nobs = z!Independent variable types: zDependent variable types: r$)r%r|r{rrzryr&r's rr)z#KDEMultivariateConditional.__repr__s5 <4<  !#'( ( 84:!%& & 03ty>>ADHH 2T_DtKK 04=@4GG &84?? r c|Sr+r,r-s rr/z#KDEMultivariateConditional.r0r c t|j}t|j}d}t |D]\}}t |}t || |j|ddf |j|jz} t ||j d| |j|ddf |j} | | z } ||| z }| S)a Returns the leave-one-out conditional likelihood of the data. If `func` is not equal to the default, what's calculated is a function of the leave-one-out conditional likelihood. Parameters ---------- bw : array_like The bandwidth parameter(s). func : callable, optional Function to transform the likelihood values (before summing); for the log likelihood, use ``func=np.log``. Default is ``f(x) = x``. Returns ------- L : float The value of the leave-one-out function for the data. Notes ----- Similar to ``KDE.loo_likelihood`, but substitute ``f(y|x)=f(x,y)/f(x)`` for ``f(x)``. rNr2) rrr~__iter__r4nextrryrzr{) rrr5yLOOxLOOr7r8Y_jr9f_yxf_xr:s rr;z)KDEMultivariateConditional.loo_likelihoods249%%49%%..00 oo  FAs4jjG#TYq!!!t_4D"&-$/"ADDDDr$*++gX%)Yq!!!t_$4 $111C*C cNAAr r c 4||j}nt||j}||j}nt||j}g}t j||f}tt j|dD]}t|j |j ||ddf|j |j z}t|j |jd|j||ddf|j }|||z t j|S)aQ Evaluate the probability density function. Parameters ---------- endog_predict : array_like, optional Evaluation data for the dependent variables. If unspecified, the training data is used. exog_predict : array_like, optional Evaluation data for the independent variables. Returns ------- pdf : array_like The value of the probability density at `endog_predict` and `exog_predict`. Notes ----- The formula for the conditional probability density is: .. math:: f(y|x)=\frac{f(x,y)}{f(x)} with .. math:: f(x)=\prod_{s=1}^{q}h_{s}^{-1}k \left(\frac{x_{is}-x_{js}}{h_{s}}\right) where :math:`k` is the appropriate kernel for each variable. Nrr2)r}r r{r~r|rrr>rrrrryrzr?r@)r endog_predict exog_predictrAr3r8rrs rrBzKDEMultivariateConditional.pdfs'<   JMM)-DDM  9LL(t|DDL |'DEE rx --a011 ' 'Adi%1!QQQ$%7"&-$/"ADDDDtwtz{{+$)$0AAA$6 $111C NN4#: & & & &z'"""r c ||j}nt||j}||j}nt||j}t j|d}t j|}t|D]}t|j |jd|j||ddf|j |j z }t j |}t|j d|j|j||ddf|jdddd}t|j |jd|j||ddf|j d }||zd } | |j |zz ||<|S) a Cumulative distribution function for the conditional density. Parameters ---------- endog_predict : array_like, optional The evaluation dependent variables at which the cdf is estimated. If not specified the training dependent variables are used. exog_predict : array_like, optional The evaluation independent variables at which the cdf is estimated. If not specified the training independent variables are used. Returns ------- cdf_est : array_like The estimate of the cdf. Notes ----- For more details on the estimation see [2]_, and p.181 in [1]_. The multivariate conditional CDF for mixed data (continuous and ordered/unordered discrete) is estimated by: .. math:: F(y|x)=\frac{n^{-1}\sum_{i=1}^{n}G(\frac{y-Y_{i}}{h_{0}}) W_{h}(X_{i},x)}{\widehat{\mu}(x)} where G() is the product kernel CDF estimator for the dependent (y) variable(s) and W() is the product kernel CDF estimator for the independent variable(s). References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Liu, R., Yang, L. "Kernel estimation of multivariate cumulative distribution function." Journal of Nonparametric Statistics (2008) Nrr2rDrErFF)rr3rrGrHrItosumrr3rrrS)r}r r{r~r|rrr\r>rrrzrr@ryr]) rrrN_data_predictrJr8mu_x cdf_endogcdf_exogSs rrKzKDEMultivariateConditional.cdfsR   JMM)-DDM  9LL(t|DDL,//2(>**~&& 0 0A ,49%1!QQQ$%7!%22248I>D:d##DTWQtz\2*7111*=&*m&4&;&5U DDDIDGDJKK0ty)5ad);%)_ECCCHX%***22Adi$./GAJJr c &t|j}d}t|j}t j|jdz df}t |D]\}}|dd|jdf}|ddd|jf} t j| |} t j|| } t j||} t j||} t||jd| |j |ddf|j d}t||jd| |j |ddf|j d}t|d|j| | |j dddd }||z|z |d zz }t|| |j|ddf |j |j z |z }t||jd| |j |ddf |j |z }|||d zz d ||z zz z }||z S) a The integrated mean square error for the conditional KDE. Parameters ---------- bw : array_like The bandwidth parameter(s). Returns ------- CV : float The cross-validation objective function. Notes ----- For more details see pp. 156-166 in [1]_. For details on how to handle the mixed variable types see [2]_. The formula for the cross-validation objective function for mixed variable types is: .. math:: CV(h,\lambda)=\frac{1}{n}\sum_{l=1}^{n} \frac{G_{-l}(X_{l})}{\left[\mu_{-l}(X_{l})\right]^{2}}- \frac{2}{n}\sum_{l=1}^{n}\frac{f_{-l}(X_{l},Y_{l})}{\mu_{-l}(X_{l})} where .. math:: G_{-l}(X_{l}) = n^{-2}\sum_{i\neq l}\sum_{j\neq l} K_{X_{i},X_{l}} K_{X_{j},X_{l}}K_{Y_{i},Y_{j}}^{(2)} where :math:`K_{X_{i},X_{l}}` is the multivariate product kernel and :math:`\mu_{-l}(X_{l})` is the leave-one-out estimator of the pdf. :math:`K_{Y_{i},Y_{j}}^{(2)}` is the convolution kernel. The value of the function is minimized by the ``_cv_ls`` method of the `GenericKDE` class to return the bw estimates that minimize the distance between the estimated and "true" probability density. References ---------- .. [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) .. [2] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) rrNFrgauss_convolutionwangryzin_convolutionaitchisonaitken_convolution)rr3rrGrIrHrrUr2)rrfloatrronesr4r{kronrr~rzryr])rrzLOOCVrexpanderrfZXYYe_LYe_RXe_LXe_RK_Xi_XlK_Xj_XlK2_Yi_YjGf_X_Ym_xs rrjzKDEMultivariateConditional.imse[se^49%% TY7DIM1-..t__ 5 5EB!!!TZ[[.!A!!![dj[.!A71h''D78Q''D71h''D78Q''D2djkk?(, "aaa%(8$(O5BBBG2djkk?(, "aaa%(8$(O5BBBGBq|,4)- %8%<%B"' )))H 7"X-2244tQw>A1"DIb!!!e4D3D#'=4?#BEEEGKLEr$*++aR%)Yr111u%5$5 $111378C 1sax<1 #44 4BBDyr c6d}|j|j|jf}||fS)rlr )r{ryrzrms rrpz/KDEMultivariateConditional._get_class_vars_types$1 j$-A :%%r r+rqrrr,r rr r Us??D2222&   '2k&&&&P2#2#2#2#hFFFFPNNN`&&&&&r r )rvnumpyrr _kernel_baserrrrr __all__r r r,r rrs< Q P Pi&i&i&i&i&ji&i&i&X Z&Z&Z&Z&Z&Z&Z&Z&Z&Z&r