M/Ph$dZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z mZmZmZddgZGd de ZGd deZGd d ZGd deZdS)aY Multivariate Conditional and Unconditional Kernel Density Estimation with Mixed Data Types References ---------- [1] Racine, J., Li, Q. Nonparametric econometrics: theory and practice. Princeton University Press. (2007) [2] Racine, Jeff. "Nonparametric Econometrics: A Primer," Foundation and Trends in Econometrics: Vol 3: No 1, pp1-88. (2008) http://dx.doi.org/10.1561/0800000009 [3] Racine, J., Li, Q. "Nonparametric Estimation of Distributions with Categorical and Continuous Data." Working Paper. (2000) [4] Racine, J. Li, Q. "Kernel Estimation of Multivariate Conditional Distributions Annals of Economics and Finance 5, 211-235 (2004) [5] Liu, R., Yang, L. "Kernel estimation of multivariate cumulative distribution function." Journal of Nonparametric Statistics (2008) [6] Li, R., Ju, G. "Nonparametric Estimation of Multivariate CDF with Categorical and Continuous Data." Working Paper [7] Li, Q., Racine, J. "Cross-validated local linear nonparametric regression" Statistica Sinica 14(2004), pp. 485-512 [8] Racine, J.: "Consistent Significance Testing for Nonparametric Regression" Journal of Business & Economics Statistics [9] Racine, J., Hart, J., Li, Q., "Testing the Significance of Categorical Predictor Variables in Nonparametric Regression Models", 2006, Econometric Reviews 25, 523-544 N)optimize) mquantiles) GenericKDEEstimatorSettingsgpke LeaveOneOut _get_type_pos _adjust_shape_compute_min_std_IQR kernel_func KernelRegKernelCensoredRegcheZdZdZ ddZd Zd Zd Zdd Zd Z dZ ddZ ddZ dZ dZdZdS)ra Nonparametric kernel regression class. Calculates the conditional mean ``E[y|X]`` where ``y = g(X) + e``. Note that the "local constant" type of regression provided here is also known as Nadaraya-Watson kernel regression; "local linear" is an extension of that which suffers less from bias issues at the edge of the support. Note that specifying a custom kernel works only with "local linear" kernel regression. For example, a custom ``tricube`` kernel yields LOESS regression. Parameters ---------- endog : array_like This is the dependent variable. exog : array_like The training data for the independent variable(s) Each element in the list is a separate variable var_type : str The type of the variables, one character per variable: - c: continuous - u: unordered (discrete) - o: ordered (discrete) reg_type : {'lc', 'll'}, optional Type of regression estimator. 'lc' means local constant and 'll' local Linear estimator. Default is 'll' bw : str or array_like, optional Either a user-specified bandwidth or the method for bandwidth selection. If a string, valid values are 'cv_ls' (least-squares cross-validation) and 'aic' (AIC Hurvich bandwidth estimation). Default is 'cv_ls'. User specified bandwidth must have as many entries as the number of variables. ckertype : str, optional The kernel used for the continuous variables. okertype : str, optional The kernel used for the ordered discrete variables. ukertype : str, optional The kernel used for the unordered discrete variables. defaults : EstimatorSettings instance, optional The default values for the efficient bandwidth estimation. Attributes ---------- bw : array_like The bandwidth parameters. llcv_lsgaussian wangryzinaitchisonaitkenNc ||_||_||_||_||_||_|jt vr|jt vr|jt vstdt|j|_ t|d|_ t||j |_ tj|j |j f|_tj|j d|_t%|j|j|_| t-n| } || t1|t2s;tj|}t||j krtd|js|||_dS|||_dS)NXuser specified kernel must be a supported kernel from statsmodels.nonparametric.kernels.rrlcrz;bw must have the same dimension as the number of variables.)var_type data_typereg_typeckertypeokertypeukertyper ValueErrorlenk_varsr endogexognp column_stackdatashapenobsdict_est_loc_constant_est_loc_linearestr _set_defaults isinstancestrasarray efficient_compute_reg_bwbw_compute_efficient) selfr#r$rrr4rrrdefaultss k/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/nonparametric/kernel_regression.py__init__zKernelReg.__init__]s! !         ,,+1M1MM[00NOO O$-(( "5!,, !$ 44 OTZ$;<< HTY''* 41d6JKKK*2*:$&&& 8$$$"c"" 9BB2ww$+%% "8999~ 2**2..DGGG--b11DGGGct|tsd|_tj|S||_|dkr|j}n|j}tj|jd}d|z|j ddtj |jdzz zz}|j |j }tj|||fd d d }|S) Nzuser-specifiedrraxisg(\?grg@@)x0argsmaxitermaxfundisp)r/r0 _bw_methodr%r1cv_loo aic_hurvichstdr$r)sizer-rrfmin)r6r4resXh0func bw_estimateds r8r3zKernelReg._compute_reg_bw}s"c"" .DO:b>> !!DOW}}k&tyq)))Atq27491+E+E+E'EFGHB8DM*D#=4(14SqJJJL r:c |j\}}t||||j|j|j|jdt |z }|ddtjf}||z }tj |j ||z} ||z d}tj |dz|dzf} | | d<|| dddf<|| dddf<| | ddddf<||z} tj |dzdf} | | d<||z | z d| dddf<tj tj | | } | d}| ddddf}||fS)a Local linear estimator of g(x) in the regression ``y = g(x) + e``. Parameters ---------- bw : array_like Vector of bandwidth value(s). endog : 1D array_like The dependent variable. exog : 1D or 2D array_like The independent variable(s). data_predict : 1D array_like of length K, where K is the number of variables. The point at which the density is estimated. Returns ------- D_x : array_like The value of the conditional mean at `data_predict`. Notes ----- See p. 81 in [1] and p.38 in [2] for the formulas. Unlike other methods, this one requires that `data_predict` be 1D. Fr' data_predictrrrrtosumNrr<rrr)r(rrrrrfloatr%newaxisdotTsumemptylinalgpinv)r6r4r#r$rQr)r"kerM12M22M ker_endogVmean_mfxmeanmfxs r8r,zKernelReg._est_loc_linears2z f2D| M M M M    #(++ .!!!RZ- \!fSUC#I&&Syoo1o%% Hfqj&1*- . .''))$!QRR%!""a%!""abb& %K Hfqj!_ % %--//$L(I5:::BB!""a%6")..++Q//{qrr111uoSyr:c t||||j|j|j|jd}t j|t j|}||zd}|d}||z }|jd} |t| z } t||||jdd} | ddt j f} || zd t| z } | d t| z } | | z || z| z z }|| z|| zz |dzz }||fS) ad Local constant estimator of g(x) in the regression y = g(x) + e Parameters ---------- bw : array_like Array of bandwidth value(s). endog : 1D array_like The dependent variable. exog : 1D or 2D array_like The independent variable(s). data_predict : 1D or 2D array_like The point(s) at which the density is estimated. Returns ------- G : ndarray The value of the conditional mean at `data_predict`. B_x : ndarray The marginal effects. FrPrr< d_gaussian)r'rQrrrRN) rrrrrr%reshaper(rXrTrU)r6r4r#r$rQker_xG_numerG_denomGr)f_xker_xcd_mxd_fxB_xs r8r+zKernelReg._est_loc_constants`.Rd"m"m"m"m """  5"(5//225=%%1%--)))## g z!}d #bt,#}+! ### 2: &$$!$,,,uT{{:  """U4[[0Sj1t8c>)~$.7A:>#v r:c ^tj|j|jf}t|jD]H}t ||j|j|ddf|j|j|j|j d|dd|f<I| d}||z }t|j |j|j |j |tdd}tj||jdf}|j |z d z dt#|jz }dtj|t#|jz zdtj|d zt#|jz z z }tj||z} | S) a  Computes the AIC Hurvich criteria for the estimation of the bandwidth. Parameters ---------- bw : str or array_like See the ``bw`` parameter of `KernelReg` for details. Returns ------- aic : ndarray The AIC Hurvich criteria, one element for each variable. func : None Unused here, needed in signature because it's used in `cv_loo`. References ---------- See ch.2 in [1] and p.35 in [2]. NF)r'rQrrrrrRrr<r2)r#r$rrr4r7rrg)r%rYr)rangerr$rrrrrXrr#rrfitrhrTtracelog) r6r4rMHjdenomgxsigmafracaics r8rFzKernelReg.aic_hurvichs( Hdi+ , ,ty!! ( (A2DIDIacN$(MDM$(MDM!&(((AaaadGG 1  I TZdi$- $ " 1E B B BDDDDGCEE!MZTYN + +*r/A%***22U495E5EEBHQKK% "2"222RXa[[1_di(8(888: fUmmd" r:c Nt|j}t|j}d}t |D]P\}}t |}|||| |j|ddf d} ||j|| z dzz }Q||jz S)af The cross-validation function with leave-one-out estimator. Parameters ---------- bw : array_like Vector of bandwidth values. func : callable function Returns the estimator of g(x). Can be either ``_est_loc_constant`` (local constant) or ``_est_loc_linear`` (local_linear). Returns ------- L : float The value of the CV function. Notes ----- Calculates the cross-validation least-squares function. This function is minimized by compute_bw to calculate the optimal value of `bw`. For details see p.35 in [2] .. math:: CV(h)=n^{-1}\sum_{i=1}^{n}(Y_{i}-g_{-i}(X_{i}))^{2} where :math:`g_{-i}(X_{i})` is the leave-one-out estimator of g(X) and :math:`h` is the vector of bandwidths rN)r#r$rQrg)r r$r#__iter__ enumeratenextr)) r6r4rMLOO_XLOO_YLiiX_not_iYrls r8rEzKernelReg.cv_loo*s:DI&&DJ''0022 $U++ + +KBU ARqx#'9RU#3"3555568A $*R.1$* *AA49}r:cLtj|j}|d}tj|}||z ||z zdz}||z dzd||z dzdz}||z S)a Returns the R-Squared for the nonparametric regression. Notes ----- For more details see p.45 in [2] The R-Squared is calculated by: .. math:: R^{2}=\frac{\left[\sum_{i=1}^{n} (Y_{i}-\bar{y})(\hat{Y_{i}}-\bar{y}\right]^{2}}{\sum_{i=1}^{n} (Y_{i}-\bar{y})^{2}\sum_{i=1}^{n}(\hat{Y_{i}}-\bar{y})^{2}}, where :math:`\hat{Y_{i}}` is the mean calculated in `fit` at the exog points. rrgr<)r%squeezer#rurcrX)r6rYhatY_barR2_numerR2_denoms r8 r_squaredzKernelReg.r_squaredSs Jtz " "xxzz!} %iD5L166881<YN''Q'//E\A%***223(""r:c |j|j}||j}nt||j}t j|d}t j|f}t j||jf}t|D]j}||j |j |j||ddf}t j |d||<t j |d}|||ddf<k||fS)a Returns the mean and marginal effects at the `data_predict` points. Parameters ---------- data_predict : array_like, optional Points at which to return the mean and marginal effects. If not given, ``data_predict == exog``. Returns ------- mean : ndarray The regression result for the mean (i.e. the actual curve). mfx : ndarray The marginal effects, i.e. the partial derivatives of the mean. NrrQr) r-rr$r r"r%r(rYrtr4r#r r6rQrMN_data_predictrcrdirbmfx_cs r8ruz KernelReg.fitks"x &  9LL(t{CCL,//2x)**h 455~&&  AtDGTZ)5ad);===Hj!--DGJx{++EC111IISyr:2Fcltj|}t|j\}}}tj||rWtj||stj||rt dt |||||}nt|||}|jS)a Significance test for the variables in the regression. Parameters ---------- var_pos : sequence The position of the variable in exog to be tested. Returns ------- sig : str The level of significance: - `*` : at 90% confidence level - `**` : at 95% confidence level - `***` : at 99* confidence level - "Not Significant" : if not significant z3Discrete variable in hypothesis. Must be continuous) r%r1r ranyr TestRegCoefC TestRegCoefDsig) r6var_posnboot nested_respivotix_contix_ordix_unordSigs r8sig_testzKernelReg.sig_tests&*W%%$1$-$@$@! 6''" # # 5vfWo&& X"&'1B*C*C X !VWWWtWeZGGCCtWe44Cwr:cd}|dt|jzdzz }|dt|jzdzz }|d|jzdzz }|d|jzdzz }|d|jzdzz }|S) Provide something sane to print.zKernelReg instance Number of variables: k_vars =  zNumber of samples: N = Variable types: BW selection method: Estimator type: r0r"r)rrDrr6rprs r8__repr__zKernelReg.__repr__s$ /#dk2B2BBTII *S^^;dBB &6== &84?? !DM1D88 r:c6d}|j|j|jf}||fS)z@Helper method to be able to pass needed vars to _compute_subset.r)rr"r)r6 class_type class_varss r8_get_class_vars_typezKernelReg._get_class_vars_types$ mT[$-@ :%%r:c<|ddddf}t|S)a^ Computes the measure of dispersion. The minimum of the standard deviation and interquartile range / 1.349 References ---------- See the user guide for the np package in R. In the notes on bwscaling option in npreg, npudens, npcdens there is a discussion on the measure of dispersion Nr)r )r6r's r8_compute_dispersionzKernelReg._compute_dispersions'AAAqrrE{#D)))r:)rrrrrNN)rrF)__name__ __module__ __qualname____doc__r9r3r,r+rFrErrurrrrr:r8rr-s..^AH/:6:2222@   *:::x///b))))V'''R###0!!!!F>&&& * * * * *r:cDeZdZdZ ddZd Zd Zd Zd Zdd Z dS)ra, Nonparametric censored regression. Calculates the conditional mean ``E[y|X]`` where ``y = g(X) + e``, where y is left-censored. Left censored variable Y is defined as ``Y = min {Y', L}`` where ``L`` is the value at which ``Y`` is censored and ``Y'`` is the true value of the variable. Parameters ---------- endog : list with one element which is array_like This is the dependent variable. exog : list The training data for the independent variable(s) Each element in the list is a separate variable dep_type : str The type of the dependent variable(s) c: Continuous u: Unordered (Discrete) o: Ordered (Discrete) reg_type : str Type of regression estimator lc: Local Constant Estimator ll: Local Linear Estimator bw : array_like Either a user-specified bandwidth or the method for bandwidth selection. cv_ls: cross-validation least squares aic: AIC Hurvich Estimator ckertype : str, optional The kernel used for the continuous variables. okertype : str, optional The kernel used for the ordered discrete variables. ukertype : str, optional The kernel used for the unordered discrete variables. censor_val : float Value at which the dependent variable is censored defaults : EstimatorSettings instance, optional The default values for the efficient bandwidth estimation Attributes ---------- bw : array_like The bandwidth parameters rraitchison_aitken_reg wangryzin_regrNc ~||_||_||_||_||_||_|jt vr|jt vr|jt vstdt|j|_ t|d|_ t||j |_ tj|j |j f|_tj|j d|_t%|j|j|_| t-n| } || | |_|j|| n tj|jdf|_|js|||_dS|||_dS)Nrrrr) rrrrrrr r r!r"r r#r$r%r&r'r(r)r*r+r,r-rr. censor_valcensoredonesW_inr2r3r4r5) r6r#r$rrr4rrrrr7s r8r9zKernelCensoredReg.__init__s ! !         ,,+1M1MM[00NOO O$-(( "5!,, !$ 44 OTZ$;<< HTY''* 41d6JKKK*2*:$&&& 8$$$$ ? & MM* % % % %A//DI~ 2**2..DGGG--b11DGGGr:c|j|kdz|_tjtj|j}||_tj|jt|_ tj t||j |<tj|j||_t|jd|_tj|j ||_ tj|j||_tj|jdf|_t#d|jdzD]}d}t#d|D];}||j|z t%|j|z dzz |j|dz zz}<||j|dz zt%|j|z dzz |j|dz df<dS)N?rr)r#dr%argsortrsortixzerosr(int sortix_revaranger!r r$rYr)rrtrT)r6rixrPrys r8rzKernelCensoredReg.censoreds* *b0 Z 4:.. / / (28S11 iB00Z 2// "4:q11 Jty}-- DF2J''Hdi^,, q$)a-(( M MAA1a[[ K Kty1}uTY'7'7'9!';EE &6== &84?? !DM1D88 r:c |j\}}t||||j|j|j|jd}||ddt jfz}||z } t j| j | |z} | |z d} t j |dz|dzf} | | d<| | dddf<| | dddf<| | ddddf<||z} t j |dzdf} | | d<||z | z d| dddf<t jt j | | }|d}|ddddf}||fS)a Local linear estimator of g(x) in the regression ``y = g(x) + e``. Parameters ---------- bw : array_like Vector of bandwidth value(s) endog : 1D array_like The dependent variable exog : 1D or 2D array_like The independent variable(s) data_predict : 1D array_like of length K, where K is the number of variables. The point at which the density is estimated Returns ------- D_x : array_like The value of the conditional mean at data_predict Notes ----- See p. 81 in [1] and p.38 in [2] for the formulas Unlike other methods, this one requires that data_predict be 1D FrPNrr<rrS)r(rrrrrr%rUrVrWrXrYrZr[)r6r4r#r$rQWr)r"r\r]r^r_r`rarbrcrds r8r,z!KernelCensoredReg._est_loc_linear:s4z f2D| M M M M 888#aaam$$\!fSUC#I&&Syoo1o%% Hfqj&1*- . .''))$!QRR%!""a%!""abb& %K Hfqj!_ % %--//$L(I5:::BB!""a%6")..++Q//{qrr111uoSyr:c t|j}t|j}t|j}d}t |D]`\}}t |} t |} ||| | |j|ddf | d} ||j|| z dzz }a||jz S)a{ The cross-validation function with leave-one-out estimator Parameters ---------- bw : array_like Vector of bandwidth values func : callable function Returns the estimator of g(x). Can be either ``_est_loc_constant`` (local constant) or ``_est_loc_linear`` (local_linear). Returns ------- L : float The value of the CV function Notes ----- Calculates the cross-validation least-squares function. This function is minimized by compute_bw to calculate the optimal value of bw For details see p.35 in [2] .. math:: CV(h)=n^{-1}\sum_{i=1}^{n}(Y_{i}-g_{-i}(X_{i}))^{2} where :math:`g_{-i}(X_{i})` is the leave-one-out estimator of g(X) and :math:`h` is the vector of bandwidths rN)r#r$rQrrg)r r$r#rrrrr)) r6r4rMrrLOO_Wrrrrwrls r8rEzKernelCensoredReg.cv_loots@DI&&DJ''0022DI&&//11 $U++ + +KBU AU ARqx#'9RU#3"3q::::;=A $*R.1$* *AA49}r:c |j|j}||j}nt||j}t j|d}t j|f}t j||jf}t|D]p}||j |j |j||ddf|j }t j |d||<t j |d}|||ddf<q||fS)zJ Returns the marginal effects at the data_predict points. Nr)rQrr) r-rr$r r"r%r(rYrtr4r#rrrs r8ruzKernelCensoredReg.fitsx &  9LL(t{CCL,//2x)**h 455~&&  AtDGTZ)5ad);"i)))Hj!--DGJx{++EC111IISyr:)rrrrrNr) rrrrr9rrr,rErurr:r8rrs,,Z#4u#E#E#E G G GGJsuuQ P aaa  Jq1c$.112 3 3CA~""$$uQxx/ r:cltj|d}tj|jf}t |jD]U}tjd||df}||df}||ddf}|||||<Vtj|} | S)z Calculates the SE of lambda by nested resampling Used to pivot the statistic. Bootstrapping works better with estimating pivotal statistics but slows down computation significantly. rr(rrHN) r%r(rYrrtrandomrandintrrG) r6rrKrrrindY1X1 se_lambdas r8rzTestRegCoefC._compute_se_lambdas HQKKNhdi\***ty!! 2 2A)##Aq1v#66C36B36B))"b11CFFF3KK r:c tj|jf}|j}t j|j}tj|d}tj|dd|j fd|dd|j f<t|||j |j j |jtdd}tj||df}||z }|tj|z }t%|jD]S}tjd||df }||df} || z} || |j||<T||_d } |jt1|d krd } |jt1|d krd} |jt1|dkrd} | S)a' Computes the significance value for the variable(s) tested. The empirical distribution of the test statistic is obtained through bootstrapping the sample. The null hypothesis is rejected if the test statistic is larger than the 90, 95, 99 percentiles. rrNr<FrsrrrNot Significant?*ffffff?**Gz?***)r%rYrr#copydeepcopyr$r(rcrrrrrr4rrurhrtrrrt_distrr) r6rrrKrr_erre_bootY_bootrs r8rzTestRegCoefC._compute_sig0s/// J M$) $ $ HQKKN!wqDN):';!DDD!!!T^  aDM4:+>05AAA C C CCF355 L Jq1a& ! ! E  Ntz"" C CA)##Aq1v#66CsAvYFZF// BBF1II  >Jvs33 3 3C >Jvt44 4 4C >Jvt44 4 4C r:N)rrF) rrrrr9rrrrrrr:r8rrs00h@C '''      $#####r:rc$eZdZdZdZdZdZdS)ra Significance test for the categorical variables in a nonparametric regression. Parameters ---------- model : Instance of KernelReg class This is the nonparametric regression model whose elements are tested for significance. test_vars : tuple, list of one element index of position of the discrete variable to be tested for significance. E.g. (3) tests variable at position 3 for significance. nboot : int Number of bootstrap samples used to determine the distribution of the test statistic in a finite sample. Default is 400 Attributes ---------- sig : str The significance level of the variable(s) tested "Not Significant": Not significant at the 90% confidence level Fails to reject the null "*": Significant at the 90% confidence level "**": Significant at the 95% confidence level "***": Significant at the 99% confidence level Notes ----- This class currently does not allow joint hypothesis. Only one variable can be tested at a time References ---------- See [9] and chapter 12 in [1]. c tjtj|jdd|jf}tj|d}t |||j|jj |j td}tj |}d|dd|jf<||d}tj||df}tj|df}|ddD]N} | |dd|jf<||d} tj| |df} || |z dzz }O|d t%|z } | S) zComputes the test statisticNrFrsrrrrgr<)r%sortuniquer$rr(rrrrr4rrrrurhrrXrT) r6rrKdom_xrrrm0zvecrm1avgs r8rzTestRegCoefD._compute_test_stat|sl $)AAAt~,=">??@@ HQKKN!Q tz/BDG%6%G%G%GIII ]1   !111dn  YYBY ' ' * ZQF # #xAqrr # #A$%Bqqq$. !++A.BBA''B R"WN "DDhhAhq) r:c|}|j}|j}tj|d}||z }|tj|z }d}d}||z}||z} |dz } tj|jdf} t|jD]i} tj | } tj dd|df}|| k}||| |<|| z}| ||| | <jd}|jt| dkrd }|jt| d krd }|jt| d krd }|S)z8Calculates the significance level of the variable testedrgP/7gw?gw@rrrrrrrrr)_est_cond_meanr#r$r%r(rcrYrrtrrruniformrrr)r6mrrKrufct1fct2u1u2rI_distryu_bootprobrrrs r8rzTestRegCoefD._compute_sigsi    ! ! J I HQKKN E  N   AX AX H 4:a.))tz"" ; ;A]2&&F9$$Qq!A$77D(CS'F3KZF//::F1II >Jvs33 3 3C >Jvt44 4 4C >Jvt44 4 4C r:ctjtj|jdd|jf|_t j|j}d}|jD]4}||dd|jf<||j |dz }5|tt|jz }tj |tj |jddf}|S)z_ Calculates the expected conditional mean m(X, Z=l) for all possible l Nrrr)r%rrr$rrrrrrurTr!rhr()r6rKrrs r8rzTestRegCoefD._est_cond_means WRYtyDN1B'CDDEE M$) $ $  5 5A$%Aaaa q11!4 4AA c$*oo&& & Jq28DI..q115 6 6r:N)rrrrrrrrr:r8rrVsM##J.   Dr:r)rrnumpyr%scipyrscipy.stats.mstatsr _kernel_baserrrr r r r r __all__rrrrrr:r8r s> ))))))QQQQQQQQQQQQQQQQQQQQ + ,]*]*]*]*]* ]*]*]*@ jjjjj jjjZYYYYYYYYxmmmmm<mmmmmr: