M/PhRdZddlmZddlZddlZddlmZddlZ ddl m cm Z ddlmZddlmZmZmZmZddlmcmZddlmZmZddlmZdd lmZdd l m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'ddl(m)Z)ddZ*GddeZ+GddeZ,e j-e,e+Gdde#eZ.Gdde#eZ/dZ0dZ1dS)za Generalized Additive Models Author: Luca Puggini Author: Josef Perktold created on 08/07/2015 )IterableN)optimize)Logit)GLM GLMResultsGLMResultsWrapper_check_convergence)PerfectSeparationError ValueWarning)cache_readonly)_is_using_pandas) matrix_sqrt)PenalizedMixin)MultivariateGamPenalty)MultivariateGAMCVPath)KFoldczt|d}|r|jnd}|t|jdd}||ddlm}t |tjrst|drJt |j tr0|j | vrtj |}ntj |j}t|}t |t }|||d}|t|kr9|s7ddl}||dt&n||}|j}|et+j|}|jd kr2|jjd ks|jjd d kr |dddf}t+j|}||fS) ztransform exog for predict using design_info Note: this is copied from base.model.Results.predict and converted to standalone function with additional options. N design_infor)dmatrixname dataframe) return_typeznan values have been dropped)r indexgetattrdatapatsyr isinstancepdSerieshasattrrstrdescribe DataFrameTlendictwarningswarnr reindexnpasarrayndimexogshape atleast_2d) modelr.r is_pandas exog_indexr orig_exog_lenis_dictr(s j/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/gam/generalized_additive_model.py_transform_predict_exogr7%s!t,,I(2dJej->> D$4!!!!!! dBI & & ,f%% ,*TY*D*D ,I!5!5!7!777|D))|D))+D T4((w{DkBBB 3t99 $ $W $ OOO! >uz!33$z/2a774=D}T""  ceZdZdZfdZ ddZdfd Zdfd ZddZ dd Z d Z dd Z e dZ e dZe dZe dZe dZxZS) GLMGamResultsa&Results class for generalized additive models, GAM. This inherits from GLMResults. Warning: some inherited methods might not correctly take account of the penalization GLMGamResults inherits from GLMResults All methods related to the loglikelihood function return the penalized values. Attributes ---------- edf list of effective degrees of freedom for each column of the design matrix. hat_matrix_diag diagonal of hat matrix gcv generalized cross-validation criterion computed as ``gcv = scale / (1. - hat_matrix_trace / self.nobs)**2`` cv cross-validation criterion computed as ``cv = ((resid_pearson / (1 - hat_matrix_diag))**2).sum() / nobs`` Notes ----- status: experimental c ||_||_||_||_|j}|dz |_|jjjd|z |_ |j|j_|j |j_ |j }|j |x|_}tj ||||fi|dS)Nrr)r1paramsnormalized_cov_paramsscaleedfsumdf_modelendogr/df_resid fittedvaluesestimate_scalesuper__init__) selfr1r<r=r>kwdsr?mu __class__s r6rGzGLMGamResults.__init__ss  %:" hllnna  (.q1C7 #m "m   !Z66r::: U 60%  ;?     r8NTcv|tj|}d}|dur!||}n||}ntj||f}ny|8t|jdr#t |j||jj\}}|;|jj|}||}ntj||f}n|}||fS)a Transform original explanatory variables for prediction Parameters ---------- exog : array_like, optional The values for the linear explanatory variables. exog_smooth : array_like values for the variables in the smooth terms transform : bool, optional If transform is False, then ``exog`` is returned unchanged and ``x`` is ignored. It is assumed that exog contains the full design matrix for the predict observations. If transform is True, then the basis representation of the smooth term will be constructed from the provided ``x``. Returns ------- exog_transformed : ndarray design matrix for the prediction NFdesign_info_linear) r+r, column_stackr!r1r7rMsmoother transform)rHr. exog_smoothrPr3ex ex_smooths r6_tranform_predict_exogz$GLMGamResults._tranform_predict_exogs,  "*[11K   "<$BB$ )<==BBGDJ8L$M$M#:Jdj&C$E$E j& J/99+FF <"BB$ ):;;BB:~r8c ||||\}}tj|fddi|}|Gt|ds7|jdkrt j||St j||S|S)a" compute prediction Parameters ---------- exog : array_like, optional The values for the linear explanatory variables exog_smooth : array_like values for the variables in the smooth terms transform : bool, optional If transform is True, then the basis representation of the smooth term will be constructed from the provided ``exog``. kwargs : Some models can take additional arguments or keywords, see the predict method of the model for the details. Returns ------- prediction : ndarray, pandas.Series or pandas.DataFrame predicted values r.rQrPrPFNpredicted_valuesr)r)rTrFpredictr!r-rr r$) rHr.rQrPkwargsrRr3predict_resultsrKs r6rXzGLMGamResults.predicts,44$AL?H5JJJ*%''/"HHHHH  !'!3+5+5 !#q((y CCCC|O:FFFF" "r8c r||||\}}tj|fddi|S)acompute prediction results Parameters ---------- exog : array_like, optional The values for which you want to predict. exog_smooth : array_like values for the variables in the smooth terms transform : bool, optional If transform is True, then the basis representation of the smooth term will be constructed from the provided ``x``. kwargs : Some models can take additional arguments or keywords, see the predict method of the model for the details. Returns ------- prediction_results : generalized_linear_model.PredictionResults The prediction results instance contains prediction and prediction variance and can on demand calculate confidence intervals and summary tables for the prediction of the mean and of new observations. rVrPF)rTrFget_prediction)rHr.rQrPrYrRr3rKs r6r\zGLMGamResults.get_predictionsS244$AL?H5JJJ&uww%bDDEDVDDDr8c:|}|jj}|j|}|jj}|t j|dz}|jdd|f}|jjj} |r/| -t j | g|f}|jj dd|f}t j ||j |} | |} | } |t j | |jjzd} t j| }| |fS)aNcontribution of a smooth term to the linear prediction Warning: This will be replaced by a predict method Parameters ---------- smooth_index : int index of the smooth term within list of smooth terms include_constant : bool If true, then the estimated intercept is added to the prediction and its standard errors. This avoids that the confidence interval has zero width at the imposed identification constraint, e.g. either at a reference point or at the mean. Returns ------- predicted : nd_array predicted value of linear term. This is not the expected response if the link function is not linear. se_pred : nd_array standard error of linear prediction rN)columnr)r1rOmask k_exog_linearr+nonzerobasisr const_idx concatenater.dotr< cov_paramsr%r@sqrt)rH smooth_indexinclude_constantvariablerOr_ start_idxidx exog_partrclinpredpartial_cov_paramscovbvarses r6partial_valueszGLMGamResults.partial_valuess0 :&}X&J, "*T**1--N111d7+ JO-  0 5.9+s!344C 3/I&DK$455!__C_88!26$ 4466;;A>> WS\\{r8Fcfddlm}m}||}|||\} } |jj} | j|j} tj | } | | } | | } | | } ||\}}|r*|j | }| |z}| | |d| | | dd|r>| | | d | zzd d | | | d | zz d d | | j|j|S) aPplot the contribution of a smooth term to the linear prediction Parameters ---------- smooth_index : int index of the smooth term within list of smooth terms plot_se : bool If plot_se is true, then the confidence interval for the linear prediction will be added to the plot. cpr : bool If cpr (component plus residual) is true, then a scatter plot of the partial working residuals will be added to the plot. include_constant : bool If true, then the estimated intercept is added to the prediction and its standard errors. This avoids that the confidence interval has zero width at the imposed identification constraint, e.g. either at a reference point or at the mean. ax : None or matplotlib axis instance If ax is not None, then the plot will be added to it. Returns ------- Figure If `ax` is None, the created figure. Otherwise, the Figure to which `ax` is connected. r) _import_mpl create_mpl_ax)ri)sblue)clwg\(\?-)r{)statsmodels.graphics.utilsrurvrsr1rO smoothersxr+argsort resid_workingscatterplot set_xlabel variable_name)rHrhplot_secprriaxrurvrjy_estrrrOr sort_indexfigresidualcpr_s r6 plot_partialzGLMGamResults.plot_partial2sk8 JIIIIIII ''9I(KK r:&  x ( *Z]] jMj!  ^-##R  %)*5H8#D JJq$!J $ $ $ 5Fq)))  9 GGAutby(#G 8 8 8 GGAutby(#G 8 8 8 h(2@AAA r8c|}|jj}|jj}t|j}|j|}|}|tj|ddz}tj |||} |j |||z} | | | S)ahypothesis test that a smooth component is zero. This calls `wald_test` to compute the hypothesis test, but uses effective degrees of freedom. Parameters ---------- smooth_index : int index of the smooth term within list of smooth terms Returns ------- wald_test : ContrastResults instance the results instance created by `wald_test` r)df_constraints) r1rOr`r&r<r_r@r+raeyer? wald_test) rHrhrjrOrkk_paramsr_ k_constraintsrl constraintsrs r6test_significancezGLMGamResults.test_significancems" :&J, t{##}X& "*T**1-a00f]Hc:: #s]':":;??AA~~k.~IIIr8rc4|j|j|j|}t j|dddf|jjz}|j|jz}|||j j z |}|S)a Compute the diagonal of the hat matrix Parameters ---------- observed : bool If true, then observed hessian is used in the hat matrix computation. If false, then the expected hessian is used. In the case of a canonical link function both are the same. This is only relevant for models that implement both observed and expected Hessian, which is currently only GLM. Other models only use the observed Hessian. _axis : int This is mainly for internal use. By default it returns the usual diagonal of the hat matrix. If _axis is zero, then the result corresponds to the effective degrees of freedom, ``edf`` for each column of exog. Returns ------- hat_matrix_diag : ndarray The diagonal of the hat matrix computed from the observed or expected hessian. )r>observedN)axis) r1hessian_factorr<r>r+rgr.r=rer%r@)rHr_axisweightswexoghess_invhds r6get_hat_matrix_diagz!GLMGamResults.get_hat_matrix_diags2*++DKtz5=,??  D)DJO;- :hll57++-- 2 2 2 > > r8c.|dS)Nr)rrrHs r6r?zGLMGamResults.edfs''a'000r8c4|jSN)hat_matrix_diagr@rs r6hat_matrix_tracezGLMGamResults.hat_matrix_traces#'')))r8c.|dS)NT)rrrs r6rzGLMGamResults.hat_matrix_diags'''666r8c<|jd|j|jz z dzz SN?rz)r>rnobsrs r6gcvzGLMGamResults.gcvs#zR$"7$)"CCaGGGr8ch|jd|jz z dz}||jz}|Sr) resid_pearsonrr@r)rHcv_s r6cvzGLMGamResults.cvs9"b4+?&?@1DIIKK ty r8)NNT)T)TFTN)Tr)__name__ __module__ __qualname____doc__rGrTrXr\rsrrrr r?rrrr __classcell__rKs@r6r:r:Ss>     .=A)-4444l!#!#!#!#!#!#FEEEEEE&&&&P11^1**^*77^7HH^H^r8r:ceZdZdS)GLMGamResultsWrapperN)rrrr8r6rrsDr8rcpeZdZdZeZeZ dfd ZdZ dfd Z ddZ ddZ ddZ xZS)GLMGama Generalized Additive Models (GAM) This inherits from `GLM`. Warning: Not all inherited methods might take correctly account of the penalization. Not all options including offset and exposure have been verified yet. Parameters ---------- endog : array_like The response variable. exog : array_like or None This explanatory variables are treated as linear. The model in this case is a partial linear model. smoother : instance of additive smoother class Examples of smoother instances include Bsplines or CyclicCubicSplines. alpha : float or list of floats Penalization weights for smooth terms. The length of the list needs to be the same as the number of smooth terms in the ``smoother``. family : instance of GLM family See GLM. offset : None or array_like See GLM. exposure : None or array_like See GLM. missing : 'none' Missing value handling is not supported in this class. **kwargs Extra keywords are used in call to the super classes. Notes ----- Status: experimental. This has full unit test coverage for the core results with Gaussian and Poisson (without offset and exposure). Other options and additional results might not be correctly supported yet. (Binomial with counts, i.e. with n_trials, is most likely wrong in pirls. User specified var or freq weights are most likely also not correct for all results.) Nrnonec | dd} d} t|dr|j|_|jj} t |d} ||||| |_| |jj} ||jj } | j d}nd} d}||_ | |_ ||_ |j|_|||_t%||j|}| dd| t)j| |jf}n|j}| g} | |j jz}| r#| !t1j||jj|}t7j|f||||||d| | s ||jdd<t|jdr|j`t|d r|j|_ d|_|`dSdS) Nhasconstrrr)alpharkpenal)rcolumns)r.familyoffsetexposurermissingformula)!getr!rrM column_namesr _handle_data data_linearxnamesr.r/r` exog_linearrO k_variables k_smooths _check_alpharrpopr+rNrb col_namesrr$ row_labelsrFrG exog_namesrrformula_linear)rHrBr.rOrrrrrrYr xnames_linearr2rr`rrrKs r6rGzGLMGam.__init__s>::j$// 4 ' ' A&*&6D # 3@M$T400  ,,UD'8LL   ,3M  */K'-a0MMKM*'  !-&&u-- &xtz1>@@@ 7D!!!  "?K#@AADD>D  M!88  00<D,<,G(.000D  AT& &$g A A9? A A A (!'DOAAA  49m , , & % 4 # # "&,D DL   r8ct|ts|gt|jjz}n$t|t st |}|S)aBcheck and convert alpha to required list format Parameters ---------- alpha : scalar, list or array_like penalization weight Returns ------- alpha : list penalization weight, list with length equal to the number of smooth terms )rrr&rOrlist)rHrs r6rzGLMGam._check_alphaGsS%** Gc$-"9:::EEE4(( KKE r8pirls:0yE> nonrobustTFc  Ht|dr|j|_|`|dvr|j|jf|||||||d| } nP| dkr%|#|j|jf|| |||||d| } | j}~ tjd||||||||| | dd | } | S)aestimate parameters and create instance of GLMGamResults class Parameters ---------- most parameters are the same as for GLM method : optimization method The special optimization method is "pirls" which uses a penalized version of IRLS. Other methods are gradient optimizers as used in base.model.LikelihoodModel. Returns ------- res : instance of wrapped GLMGamResults r)rirls) start_paramsmaxitertolr>cov_typecov_kwdsuse_trN) rrmethodrr>rrr full_outputdispmax_start_irlsr) r!rrlower _fit_pirlsrr<rFfit)rHrrrrr>rrrrrrrYresrKs r6rz GLMGam.fit\s#( 4 # # "&,D  <<>>. . .!$/$*9<*1s%+3h(-992899CC !!|';%dodj=|.<#,1/7(,1 ==6< == #z %''+(<&-f"%U'/($)*5D-. ((!'((C r8dc v|j} |j} |j|} | j\} }| t jdg| z|_n| |_t|dsd|_ ||_ d|_ |5|j | }|j |}n7t j| ||j z}|j |}|j | |}t%d|gt j|g}d}|d }|dkrL|j |}|||_ t+j||d}d}t/|D]4}|j|j |z|_||j j|| |z zz|j z }t7|| | |j}t j| |j}||j z }|j |}||||}| j dkr)t j!|| z drd }tE|tG|||d}|rn6||_$|||_ tK||j|j&|j ||| }d |_'|dz|d <||_(||_)tU|S)z:fit model with penalized reweighted least squares rNr_offset_exposurerr)r<devianceFrz2Perfect separation detected, results not available)rrrPIRLS iteration)+rBr.rpenalty_matrixr/r+array data_weightsr!r scaletyper>r starting_murXrefittedrr'infrElmRegressionResultsrangerlinkderiv penalized_wlsr<ravel_update_historysqueezer-allcloser r rJr:r=r fit_history convergedr)rHrrrrr>rrrrrBwlsexogspl_sr n_columnsrJlin_preddevhistoryr criterion wls_resultsrwlsendogmsg glm_resultss r6rzGLMGam._fit_pirlss; ) )))66!-i ? "" 5 5D   'D t/00 &$%D !  ((//B{**2..HHvg|44t7LLH##H--Bk""5"--t\2bfc]KKK J'  a<<##H--B,,R00DJ.t\4HHKIw  I ,t{/B/B2/F/FFDL!4;#3#9#9"#=#=#LL/0H('5$,OOKvg{'9::@@BBH - -H##H--B **;GDDG}}#q((R[eQ-G-G(J,S111+9iaHHI  ((,, #D+*<$/$E$(J-5*/ 111 % (1} ")  ) #K000r8aic basinhoppingc j}j}tjj}|t jj}nt jd|z}igd<|gd<gd<fd} |dkrCtdd d } | |tj | |fi| } | d } n}|d krGtdddddd} | |tj | |fi| } | j } n0|dkrtj| |fi|} | j } ntddd =t j| } |_|_|_| | fS)a find alpha by minimizing results criterion The objective for the minimization can be results attributes like ``gcv``, ``aic`` or ``bic`` where the latter are based on effective degrees of freedom. Warning: In many case the optimization might converge to a local optimum or near optimum. Different start_params or using a global optimizer is recommended, default is basinhopping. Parameters ---------- criterion='aic' name of results attribute to be minimized. Default is 'aic', other options are 'gcv', 'cv' or 'bic'. start_params : None or array starting parameters for alpha in the penalization weight minimization. The parameters are internally exponentiated and the minimization is with respect to ``exp(alpha)`` start_model_params : None or array starting parameter for the ``model._fit_pirls``. method : 'basinhopping', 'nm' or 'minimize' 'basinhopping' and 'nm' directly use the underlying scipy.optimize functions `basinhopping` and `fmin`. 'minimize' provides access to the high level interface, `scipy.optimize.minimize`. fit_kwds : keyword arguments additional keyword arguments will be used in the call to the scipy optimizer. Which keywords are supported depends on the scipy optimization function. Returns ------- alpha : ndarray penalization parameter found by minimizing the criterion. Note that this can be only a local (near) optimum. fit_res : tuple results returned by the scipy optimization routine. The parameters in the optimization problem are `log(alpha)` history : dict history of calls to pirls and contains alpha, the fit criterion and the parameters to which pirls converged to for the given alpha. Notes ----- In the test cases Nelder-Mead and bfgs often converge to local optima, see also https://github.com/statsmodels/statsmodels/issues/5381. This does not use any analytical derivatives for the criterion minimization. Status: experimental, It is possible that defaults change if there is a better way to find a global optimum. API (e.g. type of return) might also change. Ng#B ;rr<rc,tj|}dd|}d|dtj|jt |S)Nr<)rrr)r+exprappendr,r<r)pares_rrrHs r6funz$GLMGam.select_penweight..fun=sq A??0A"0E)*#,,D G  # #A & & & H  $ $RZ %<%< = = =4++ +r8nmTri)rrmaxfunrrz Nelder-Meadri)rmaxfev)roptions )minimizer_kwargsniterminimizezmethod not recognized)r>rcopyrr+zerosrlogr'updaterfminrrr& ValueErrorr)rHrrstart_model_paramsrfit_kwds scale_keepscaletype_keep alpha_keeprrIfit_resoptrrs`` @r6select_penweightzGLMGam.select_penweightsvZ Ytz**  8DN33LL6%,"677L/0!  , , , , , , , T>>D$tDDDD KK ! ! !mC>>>>G!*CC ~ % %M/2c#B#B*D*D """D KK ! ! !+CFFFFG)CC z ! !'\FFXFFG)CC455 5 H a s   ' gw&&r8 c |d}| fdt|jD}|t|d}t|j|t ||j|j|}|}|j |fS)aofind alphas by k-fold cross-validation Warning: This estimates ``k_folds`` models for each point in the grid of alphas. Parameters ---------- alphas : None or list of arrays cv_iterator : instance instance of a cross-validation iterator, by default this is a KFold instance cost : function default is mean squared error. The cost function to evaluate the prediction error for the left out sample. This should take two arrays as argument and return one float. k_folds : int number of folds if default Kfold iterator is used. This is ignored if ``cv_iterator`` is not None. Returns ------- alpha_cv : list of float Best alpha in grid according to cross-validation res_cv : instance of MultivariateGAMCVPath The instance was used for cross-validation and holds the results Notes ----- The default alphas are defined as ``alphas = [np.logspace(0, 7, k_grid) for _ in range(k_smooths)]`` Ncftj||z t|z Sr)r+linalgnormr&)x1x2s r6costz+GLMGam.select_penweight_kfold..costs%y~~b2g..R88r8c<g|]}tjddS)r)r+logspace).0_k_grids r6 z1GLMGam.select_penweight_kfold..s'OOOAbk!Q//OOOr8T)k_foldsshuffle)rOalphasgamr=rBr. cv_iterator) rrrrrOrrBrralpha_cv)rHrGrIr=rErCgam_cv gam_cv_ress ` r6select_penweight_kfoldzGLMGam.select_penweight_kfoldbsD < 9 9 9 >OOOOt~9N9NOOOF  >>>K& f+1DJ,0,<3>@@@ZZ\\ "J..r8)NNrNNNr) NrrrNrNNTFr)NrrNrNNN)rNNr)NNNr5r6)rrrrr:_results_classr_results_class_wrapperrGrrrr4rMrrs@r6rrs((T#N1HL5;EEEEEEN*HLCG9:111111hEIJNa1a1a1a1F>B,0 .m'm'm'm'^JN132/2/2/2/2/2/2/2/r8rc"eZdZdZfdZxZS)LogitGamzGeneralized Additive model for discrete Logit This subclasses discrete_model Logit. Warning: not all inherited methods might take correctly account of the penalization not verified yet. ct|ts*tj|gt |jz}||_||_d|_t||}tj ||j g|Rd|i|dS)Nrrr) rrr+rr&rrOr pen_weightrrFrGrb)rHrBrOrargsrYrrKs r6rGzLogitGam.__init__s%** @HeWs8+='>'>>??E   &xu===MdMMMeMfMMMMMr8)rrrrrGrrs@r6rQrQsK N N N N N N N N Nr8rQc|||}}}t||d|z|\}}} tj||| } | j| _| S)a weighted least squares with quadratic penalty Parameters ---------- endog : ndarray response or endogenous variable exog : ndarray design matrix, matrix of exogenous or explanatory variables penalty_matrix : ndarray, 2-Dim square penality matrix for quadratic penalization. Note, the penalty_matrix is multiplied by two to match non-pirls fitting methods. weights : ndarray weights for WLS Returns ------- results : Results instance of WLS rz)make_augmented_matrixrWLSrr<r) rBr.rryrrxaug_yaug_x aug_weightsrs r6rrsm&T>!qA 5aAE7 K KE5+&{337799K %+1133K r8cF|||}}}|jd}t|}tj||g} | jd} tjdg| z} || d|<tjdg|jdz} tj|| g} | | | fS)aMaugment endog, exog and weights with stochastic restriction matrix Parameters ---------- endog : ndarray response or endogenous variable exog : ndarray design matrix, matrix of exogenous or explanatory variables penalty_matrix : ndarray, 2-Dim square penality matrix for quadratic penalization weights : ndarray weights for WLS Returns ------- endog_aug : ndarray augmented response variable exog_aug : ndarray augmented design matrix weights_aug : ndarray augmented weights for WLS rgNr)r/rr+vstackrrd)rBr.rrrXrrxrrsr; n_samp1es_x1y1id1w1s r6rVrVs.dN!qA 71:D QB Ar7  B8A;L 2$% & &BBuuI (B4"(1+% & &C # ' 'B r2:r8r)2rcollections.abcrr'numpyr+scipyrpandasrstatsmodels.base.wrapperbasewrapperwrap#statsmodels.discrete.discrete_modelr+statsmodels.genmod.generalized_linear_modelrrrr #statsmodels.regression.linear_model regression linear_modelrstatsmodels.tools.sm_exceptionsr r statsmodels.tools.decoratorsr statsmodels.tools.datar statsmodels.tools.linalgrstatsmodels.base._penalizedrstatsmodels.gam.gam_penaltiesr9statsmodels.gam.gam_cross_validation.gam_cross_validationr5statsmodels.gam.gam_cross_validation.cross_validatorsrr7r:rpopulate_wrapperrrQrrVrr8r6rys%$$$$$ '''''''''555555<<<<<<<<<<<<000000000;;;;;;;;777777333333000000666666@@@@@@GGGGGG++++\uuuuuJuuup      ,   *M:::B/B/B/B/B/^SB/B/B/JNNNNN~uNNN,B&&&&&r8