M/PhodZddlZddlZddlZddlmcmZddl m Z ddl Z ddl m Z ddlmZddlmZddlZGddZGd d eZd ZGd d ejZGddejZdS)ah This module implements maximum likelihood-based estimation (MLE) of Gaussian regression models for finite-dimensional observations made on infinite-dimensional processes. The ProcessMLE class supports regression analyses on grouped data, where the observations within a group are dependent (they are made on the same underlying process). One use-case is repeated measures regression for temporal (longitudinal) data, in which the repeated measures occur at arbitrary real-valued time points. The mean structure is specified as a linear model. The covariance parameters depend on covariates via a link function. N)OLS)minimize)summary2) approx_fprimeceZdZdZdZdZdS)ProcessCovariancea A covariance model for a process indexed by a real parameter. An implementation of this class is based on a positive definite correlation function h that maps real numbers to the interval [0, 1], such as the Gaussian (squared exponential) correlation function :math:`\exp(-x^2)`. It also depends on a positive scaling function `s` and a positive smoothness function `u`. ct)a Returns the covariance matrix for given time values. Parameters ---------- time : array_like The time points for the observations. If len(time) = p, a pxp covariance matrix is returned. sc : array_like The scaling parameters for the observations. sm : array_like The smoothness parameters for the observation. See class docstring for details. NotImplementedErrorselftimescsms i/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/regression/process_regression.pyget_covzProcessCovariance.get_cov' "!ct)a The Jacobian of the covariance with respect to the parameters. See get_cov for parameters. Returns ------- jsc : list-like jsc[i] is the derivative of the covariance matrix with respect to the i^th scaling parameter. jsm : list-like jsm[i] is the derivative of the covariance matrix with respect to the i^th smoothness parameter. r r s rjaczProcessCovariance.jac8rrN__name__ __module__ __qualname____doc__rrrrrrs<"""""""""rrceZdZdZdZdZdS)GaussianCovarianceu] An implementation of ProcessCovariance using the Gaussian kernel. This class represents a parametric covariance model for a Gaussian process as described in the work of Paciorek et al. cited below. Following Paciorek et al [1]_, the covariance between observations with index `i` and `j` is given by: .. math:: s[i] \cdot s[j] \cdot h(|time[i] - time[j]| / \sqrt{(u[i] + u[j]) / 2}) \cdot \frac{u[i]^{1/4}u[j]^{1/4}}{\sqrt{(u[i] + u[j])/2}} The ProcessMLE class allows linear models with this covariance structure to be fit using maximum likelihood (ML). The mean and covariance parameters of the model are fit jointly. The mean, scaling, and smoothing parameters can be linked to covariates. The mean parameters are linked linearly, and the scaling and smoothing parameters use an log link function to preserve positivity. The reference of Paciorek et al. below provides more details. Note that here we only implement the 1-dimensional version of their approach. References ---------- .. [1] Paciorek, C. J. and Schervish, M. J. (2006). Spatial modeling using a new class of nonstationary covariance functions. Environmetrics, 17:483–506. https://papers.nips.cc/paper/2350-nonstationary-covariance-functions-for-gaussian-process-regression.pdf c\tj||}tj||dz }||z|z }tj| dz tj|z }|tj||dzz}|tj||z}|S)N?)npsubtractouteraddexpsqrt)r rrrdadsqmatcms rrzGaussianCovariance.get_covns [  tT * * V\\"b ! !A %Bw| VTEAI   , bhr2$$ bhr2 rctj||}tj||dz }tj|}||z}||z }t |} tj| dz } tj||dz} | | z|z } | |dzz } tj| | f}tj| | f}tj||}g}t|D]\}}|dz}||ddfxxdz cc<|dd|fxxdz cc<d|z|z }d| z| z|z}||z | |z|z z }| |z }d|dzz||dzz }|dz}|||ddf<||dd|f<d||dzz |||f<||z|| zz}||z}| |g}tdt |D]d}tj| | f}| |ddf|z||ddf<|dd|fxx| dd|f|zz cc<| |e||fS)Nr r!r?gg?) r"r#r$r%r'lenr&zeros enumerateappendrange)r rrrr(r)sdsdaar*peqmsm4cmxdq0difisccjsmi_dbottomdtopbcvjscs rrzGaussianCovariance.jaczs [  tT * * V\\"b ! !A %gbkk2gRx IIfdUQYhr2$Ci#odRUl Xq!f   Xq!f  hr2bMM  DAq !GB q!!!tHHHOHHH qqq!tHHHOHHHBhnG#:#b(Ds S7]R//Ac Ar4x"Q%+-A !GBBq!!!tHBqqq!tHRUCZ'Bq!tHBS A HA JJqMMMMq#b''""  A!Q  A!QQQ$i"nAadG aaadGGGs111a4y2~ %GGG JJqMMMMCxrNrrrrrrJs=!!F   +++++rrcLt||jd|jd|jdt|t|g}| ||jdt|t |krd}t |dS)Nrzz'ProcessMLE.__init__.. sAAAqhlAAArcg|]}d|zS)zScale%drrXs rr[z'ProcessMLE.__init__..sIIIy1}IIIrcg|]}d|zS)zSmooth%drrXs rr[z'ProcessMLE.__init__..sKKK!zA~KKKrcg|]}d|zS)zNoise%drrXs rr[z'ProcessMLE.__init__..sMMMQ9q=MMMrF)super__init__ _has_noisehasattrlistrVr2rGdata param_namesrcovrR collections defaultdictr0r1 _groups_ixverboserLk_exogrMk_scalerNk_smoothrOk_noise)r rKrLrMrNrOrrPrgkwargsxnames groups_ixr>g __class__s rrazProcessMLE.__init__s    "#!     %D0 4 # # B$,''FFAAE$*Q-,@,@AAAF :y ) ) J d:-.. .FF IIeJ4DQ4G.H.HIII IF ; * * L d;.// /FF KKu[5Fq5I/J/JKKK KF ? Nz9-- N$z1222MM% 8H8K2L2LMMMM &  ;$&&CE4[*& " " " +D11 f%% # #DAq aL   " " " "# ioa( ,Q/ (.q1 ? 4?03DLLL 4 4rc |jj}d}||||jz}||jz }||||jz}||jz }||||jz}|jr||jz }||||jz}ng}||||fS)Nr)rerfrlrmrnrbro)r rqq mean_names scale_names smooth_names noise_namess r_split_param_nameszProcessMLE._split_param_names8s& Aa mO,  T[Qq~-.  T\a$-/0 ?   A 1T\>!12KKK; kAArc \d|vr |d}ntdd|vr |d}ntdd|vr |d} nd} d|vr |d} ntdd|vr |d} ntd |tjd |tjd t| trt j|| } t| trt j|| } tj||} | j } | j }t j| } tj||}|j }|j }t j|}| 8tj| |}|j }|j }t j|}n ddgdf\}}}}t ||d| ||| | }| |j _||j _|jr ||j _|j|z|z|z|j _|S) N scale_formulaz$scale_formula is a required argumentsmooth_formulaz%smooth_formula is a required argument noise_formularztime is a required argumentrPzgroups is a required argumentz'subset' is ignoredz'drop_cols' is ignored)resubsetrMrNrOrrP)rJwarningswarn isinstancestrr"asarraypatsydmatrix design_info column_namesr` from_formularescale_design_infosmooth_design_inforbnoise_design_info exog_namesrf)clsformularer drop_colsargsrpr}r~rrrPrMrrxrNrryrOrrzmodrts rrzProcessMLE.from_formulaIss f $ $"?3MMCDD D v % %#$45NNDEE E f $ $"?3MM M V  &>DD:;; ; v  H%FF<== =   M/ 0 0 0  M2 3 3 3 dC  *:d4j))D fc " " .ZV --F]=$77 &2'4 Z ++ mND99 (4)6 j--  $}d;;J * 6 +8KJ//JJdB$ CJ); gg"" !#!#&7"&8# > ;):CH & # < ,!-/:!; rc|jjd}|d|}|jjd}||||z}|jjd}|||z||z|z}|||z|zd}||||fS)z@ Split the packed parameter vector into blocks. r\rN)rLrGrMrN) r zpmmnparpvscparpssmparnopars runpackzProcessMLE.unpacks Y_Q !B$_ "1 %"R"W*   #A &"r'"r'B,&'"r'B,-- eUE))rc:t|j|j}|}|jjd|jjdz}|jr||jjdz }tj |j tj |fS)Nr\) rrKrLfitrMrGrNrbrOr" concatenateparamsr/)r modelresultms r _get_startzProcessMLE._get_startsDJ ** O !! $t'7'=a'@ @ ? * &q) )A~v}bhqkk:;;;rc ||\}}}}|jtj|j|z }tjtj|j|}tjtj|j|}|jr,tjtj|j |} d} |j D]\} } |j |j| || || } |jr/| jdd| jddzxx| | dzz cc<|| }| dtj| dzz} | dtj|tj| |zz} |jrt+d| | S)a Calculate the log-likelihood function for the model. Parameters ---------- params : array_like The packed parameters for the model. Returns ------- The log-likelihood value at the given parameter point. Notes ----- The mean, scaling, and smoothing parameters are packed into a vector. Use `unpack` to access the component vectors. gNrr\r r-zL=)rrKr"dotrLr&rMrNrbrOrjitemsrgrrflatrGlinalgslogdetsolverkprint)r rrrrrresidrrnollr?ixr+res rloglikezProcessMLE.loglikes&&*[[%8%8"ueU RVDIu555VBF4?E22 3 3VBF4+U33 4 4 ? 8t6677B_**,, < jqsmxsnobms& rrzProcessMLE.scoresw&&*[[%8%8"ueUZZUSZZB RVDIu555VBF4?E22 3 3VBF4+U33 4 4 ? 8t6677BUc%jj03u::=E JKK_**,,2 P2 PEArb6Db6DBiGYr]FYr111u%F?2qqq51L ,RU3M!!&$55B 8"v#r111u5 ))"(1+/)***bfai7***)--##CfdD99JD$)//"g..C !B$KKK26&(C00 0KKK'7++BR((Bryr24000Bqqq$w-,.C!"  M M1VDGbL))bbj!!!R#ad)^3!!!bbj!!!S26$q'C-+@+@%@3q!!!t9%LL!!!!qqq$w--/C!"  B B1VDGbL))b2gb2gl*+++rC111I~=+++b2gb2gl*+++rvd1gm444s1aaa4y@B++++ P111d7mQ&5b2glmm$$$sx8I8I"(1+/8I/J/2)4)44$$$VCC11b2glmm$$$rw7H7H!q7H/I3(O(OO$$$ < : &"'"&"7"788 9 9 9 rc0t||j}|SN)rr)r rhesss rhessianzProcessMLE.hessianWsVTZ00 rc d|vr |d_i}d|vr|d}|}t|tr|g}n|ddg}t |D] \}}|dvrfd}nd}|5t j|r||d<n||t|z|d<tfd |||| } | j ssd } |4| d t j t j | j d zzz } |t|dz kr| d||dzzz } tj| t j| jr| j}| j} t j|  } n#t*$rd} YnwxYwGdd} | }| j|_| |_| |_d|_t5|} | S)a Fit a grouped Gaussian process regression using MLE. Parameters ---------- start_params : array_like Optional starting values. method : str or array of str Method or sequence of methods for scipy optimize. maxiter : int The maximum number of iterations in the optimization. Returns ------- An instance of ProcessMLEResults. rk minim_optsNpowellbfgs)rc0| Sr)rxr s rrzProcessMLE.fit..jacs JJqMM>)rmaxiterc0| Sr)rrs rz ProcessMLE.fit..s4<<??*r)methodx0roptionszFitting did not convergez, |gradient|=%.6fr r\z, trying %s next...ceZdZdS)ProcessMLE.fit..rsltN)rrrrrrrsltrs Drr)rkrrrr0r"isscalarr.rsuccessr'rrrrisfiniterallrrr Exceptionrnormalized_cov_params optim_retvalsscaleProcessMLEResults)r start_paramsrrrprrZmethrfrQr cov_paramsrrs` rrzProcessMLE.fit\s&   !),DL 6 ! ! -J  ??,,L fc " " (XFF ^'F (( # #GAt;&&******";w''F,3Jy)),3AG 4D,EJy)****" $$$A9 #0?.q9I9I1J1JJJCs6{{Q&&06!A#;>>C c"""{13##%% # s ||AC   )-----JJ   JJJ          DFF3", q)) s F// F>=F>ct|jds+tj||}tj||}nt j|jj|}t j|jj|} tjtj||}tjtj| |}|j |||S)a+ Returns a Gaussian process covariance matrix. Parameters ---------- time : array_like The time points at which the fitted covariance matrix is calculated. scale_params : array_like The regression parameters for the scaling part of the covariance structure. smooth_params : array_like The regression parameters for the smoothing part of the covariance structure. scale_data : DataFrame The data used to determine the scale parameter, must have len(time) rows. smooth_data : DataFrame The data used to determine the smoothness parameter, must have len(time) rows. Returns ------- A covariance matrix. Notes ----- If the model was fit using formulas, `scale` and `smooth` should be Dataframes, containing all variables that were present in the respective scaling and smoothing formulas used to fit the model. Otherwise, `scale` and `smooth` should contain data arrays whose columns align with the fitted scaling and smoothing parameters. The covariance is only for the Gaussian process and does not include the white noise variance. r) rcrer"rrrrrr&rgr) r r scale_params smooth_params scale_data smooth_datascasmorrs r covariancezProcessMLE.covariancesNty"566 4&\22C&m44CCty:JGGBty;[IIB&L1122C&M2233Cxc3///rc||j}n4t|jdrtj|jj|}t ||jdkr|d|jd}tj ||S)ao Obtain predictions of the mean structure. Parameters ---------- params : array_like The model parameters, may be truncated to include only mean parameters. exog : array_like The design matrix for the mean structure. If not provided, the model's design matrix is used. Nrr\r) rLrcrerrrr.rGr"r)r rrLrrps rpredictzProcessMLE.predictsy <9DD TY . . >=!6==D v;;A & &AdjmO,FvdF###rr)NN)NNN)rrrrrar{ classmethodrrrrrrrrr __classcell__rts@rrTrTs,11vE4E4E4E4E4E4NBBB"!# SSSSS[Sj***. < < <444l\\\| SSSSj000000d$$$$$$$$rrTc>eZdZdZfdZd dZdZdZd d ZxZ S) rz? Results class for Gaussian process regression models. cJt||||j}|d|_|d|_|d|_|d|_|jj dt|jz |_ |j j j d|_|j jj d|_|j jj d|_|j|_|jr|j jj d|_dSdS)Nrr\r )r`rarr mean_paramsrr no_paramsrKrGr.df_residrrLrlrMrmrNrnrbrOro)r rmlefitparts rrazProcessMLEResults.__init__s  6   \\&- ( (a5qEUA )!,s6=/A/AA jo+A. z,215  .4Q7 *   ::06q9DLLL : :rNTc|stjdt|dkst|dkrtjd|j|j|S)Nz''transform=False' is ignored in predictrz$extra arguments ignored in 'predict')rrr.rrr)r rL transformrrps rrzProcessMLEResults.predictsd E MC D D D t99q==CKK!OO M@ A A Az!!$+t444rcR|j||j|j||S)a Returns a fitted covariance matrix. Parameters ---------- time : array_like The time points at which the fitted covariance matrix is calculated. scale : array_like The data used to determine the scale parameter, must have len(time) rows. smooth : array_like The data used to determine the smoothness parameter, must have len(time) rows. Returns ------- A covariance matrix. Notes ----- If the model was fit using formulas, `scale` and `smooth` should be Dataframes, containing all variables that were present in the respective scaling and smoothing formulas used to fit the model. Otherwise, `scale` and `smooth` should be data arrays whose columns align with the fitted scaling and smoothing parameters. )rrrr)r rrsmooths rrzProcessMLEResults.covariance"s2:z$$T4+<%)%7HH Hrc|jj|}t|dkr!dt|z}t ||jj|ddf}|jj|ddf}|j\}}}}tj ||}tj ||}|jj |} |j | |j |j ||S)NrzGroup '%s' does not exist)rV)rrjr.rrJrMrNr{pd DataFramerrrr) r grouprrQrrr?rxryrs rcovariance_groupz"ProcessMLEResults.covariance_groupBsZ "5 ) r77a<<-E :CS// !Z*2qqq51 j,RU3 *.**G*G*I*I'; a\*kBBB l; EEE zr"z$$T%)%6%)%7%/%0 22 2r皙?cbtj}dg|jzdg|jzzdg|jzz}|jr|dg|jzz }||d<|j|d< tj tj | |d<n#t$rtj |d<YnwxYwdd lm}|j|dz |d <d |tj|jz|d <|d |d z z }|j||dzz |d|d z z<|j||dzz|dd |d z z z<|jjj|_t3j} |d}| || || S)NMeanScaleSmoothSDTypecoefzstd errr)normtvaluesr zP>|t|r\z[%.3fz%.3f]z#Gaussian process regression results)r r rlrmrnrbrorr"r'diagrrnanscipy.stats.distributionsrrsfabsrppfrrerfindexrSummary add_titleadd_df) r ynamexnametitlealphadftyprrsumms rsummaryzProcessMLEResults.summaryZs \^^x$+% DL(@@zDM)* ? ) D6DL( (C6 [6  #GBGDOO,=,=$>$>??ByMM # # #FByMMM # 322222"Y-/9 $''"&"4"45557 HHQ] # #$&Ga"Y-.?$?7eai !(*!bm2C(C7a%!)m $%:?.!! =9E u B s;BB43B4)NT)NNNr ) rrrrrarrr r(rrs@rrrs:::::,5555HHH@2220!!!!!!!!rr)rnumpyr"pandasr rstatsmodels.base.modelbaser#statsmodels.regression.linear_modelrrhscipy.optimizerstatsmodels.iolibrstatsmodels.tools.numdiffrrrrrRLikelihoodModelrTGenericLikelihoodModelResultsrrrrr3s   %%%%%%%%%333333######&&&&&&333333+"+"+"+"+"+"+"+"\[[[[[*[[[|*}$}$}$}$}$%}$}$}$@~~~~~:~~~~~r