M/PhQdZddlZddlmZddlmZddlm Z GddZ GddZ Gd d eZ dS) ax Mixed effects models Author: Jonathan Taylor Author: Josef Perktold License: BSD-3 Notes ----- It's pretty slow if the model is misspecified, in my first example convergence in loglike is not reached within 2000 iterations. Added stop criteria based on convergence of parameters instead. With correctly specified model, convergence is fast, in 6 iterations in example. N)LikelihoodModelResults)cache_readonlyc`eZdZdZdZdZdZdZdZdZ dZ d Z d Z dd Z ddZddZd S)Unita  Individual experimental unit for EM implementation of (repeated measures) mixed effects model. 'Maximum Likelihood Computations with Repeated Measures: Application of the EM Algorithm' Nan Laird; Nicholas Lange; Daniel Stram Journal of the American Statistical Association, Vol. 82, No. 397. (Mar., 1987), pp. 97-105. Parameters ---------- endog : ndarray, (nobs,) response, endogenous variable exog_fe : ndarray, (nobs, k_vars_fe) explanatory variables as regressors or fixed effects, should include exog_re to correct mean of random coefficients, see Notes exog_re : ndarray, (nobs, k_vars_re) explanatory variables or random effects or coefficients Notes ----- If the exog_re variables are not included in exog_fe, then the mean of the random constants or coefficients are not centered. The covariance matrix of the random parameter estimates are not centered in this case. (That's how it looks to me. JP) cT||_||_||_|jd|_dS)Nr)YXZshapen)selfendogexog_feexog_res _/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/sandbox/panel/mixed.py__init__z Unit.__init__=s(Qctj|j|dzztj|jtj||jjz|_dS)zcovariance of observations (nobs_i, nobs_i) (JP check) Display (3.3) from Laird, Lange, Stram (see help(Unit)) N)npidentityr dotr TS)r Dsigmas r _compute_SzUnit._compute_SDsJ+df%%q0&468!4!4556rcBtj|j|_dS)zinverse covariance of observations (nobs_i, nobs_i) (JP check) Display (3.2) from Laird, Lange, Stram (see help(Unit)) N)LinvrWr s r _compute_WzUnit._compute_WKstvrctj|j|j}|jtjtj|||jz |_dS)zprojection matrix (nobs_i, nobs_i) (M in regression ?) (JP check, guessing) Display (3.10) from Laird, Lange, Stram (see help(Unit)) W - W X Sinv X' W' N)rrr!r rP)r Sinvts r compute_PzUnit.compute_PQsC F4646 " ""&4!#666rcT|jtj|j|z |_dS)zoresidual after removing fixed effects Display (3.5) from Laird, Lange, Stram (see help(Unit)) N)rrrr r)r alphas r _compute_rzUnit._compute_rZs$ "&///rc tj|tjtj|jj|j|j|_dS)zcoefficients for random effects/coefficients Display (3.4) from Laird, Lange, Stram (see help(Unit)) D Z' W r N)rrr rr!r*b)r rs r _compute_bzUnit._compute_bas; 26"&46":":DFCCDDrc||||||||dS)z Compute unit specific parameters in Laird, Lange, Stram (see help(Unit)). Displays (3.2)-(3.5). N)rr#r,r/)r arrs rfitzUnit.fitisR 5!!!   rcptjtj|j|j|jS)zU Utility function to compute X^tWY (transposed ?) for Unit instance. )rrr!rr r"s r compute_xtwyzUnit.compute_xtwyvs(vbfTVTV,,df555rcztjtj|jj|j|jS)zF Utility function to compute X^tWX for Unit instance. )rrr rr!r"s r compute_xtwxzUnit.compute_xtwx|s*vbfTVXtv..777rNc|||tj|j|}|tjtj|j|j|z S)aa Approximate covariance of estimates of random effects. Just after Display (3.10) in Laird, Lange, Stram (see help(Unit)). D - D' Z' P Z D Notes ----- In example where the mean of the random coefficient is not zero, this is not a covariance but a non-centered moment. (proof by example) )r(rrr rr%)r rr&r's r cov_randomzUnit.cov_randomsV   NN4 F461  26"&df--q1111rFc|rhtjtj|j|jtj|j|jzz dz S|td|j tj|j |z }tjtj|j|tj|j|zz dz S)a Individual contributions to the log-likelihood, tries to return REML contribution by default though this requires estimated fixed effect a to be passed as an argument. no constant with pi included a is not used if ML=true (should be a=None in signature) If ML is false, then the residuals are calculated for the given fixed effects parameters a. g@Nz;need fixed effect a for REML contribution to log-likelihood) rlogrdetr!r*rsum ValueErrorrr )r r1MLr*s rlogLz Unit.logLs  PF15==))TVbfTVTV6L6L-L,Q,Q,S,SSWYY Yy !^___***AF15==))Q1B1B-B,G,G,I,IIRO Orc4d||zSz@deviance defined as 2 times the negative loglikelihood r>r?r r>s rdeviancez Unit.deviancesTYY"Y%%%%rNF)__name__ __module__ __qualname____doc__rrr#r(r,r/r2r4r6r8r?rFrrrrsD   777 777000EEE   666 888 2222"PPPP*&&&&&&rrceZdZdZdZdZddZddZdZdZ e d Z e d Z d Z e d Zdd ZddZdZddZddZdS) OneWayMixeda Model for EM implementation of (repeated measures) mixed effects model. 'Maximum Likelihood Computations with Repeated Measures: Application of the EM Algorithm' Nan Laird; Nicholas Lange; Daniel Stram Journal of the American Statistical Association, Vol. 82, No. 397. (Mar., 1987), pp. 97-105. Parameters ---------- units : list of units the data for the individual units should be attached to the units response, fixed and random : formula expression, called as argument to Formula *available results and alias* (subject to renaming, and coversion to cached attributes) params() -> self.a : coefficient for fixed effects or exog cov_params() -> self.Sinv : covariance estimate of fixed effects/exog bse() : standard deviation of params cov_random -> self.D : estimate of random effects covariance params_random_units -> [self.units[...].b] : random coefficient for each unit *attributes* (others) self.m : number of units self.p : k_vars_fixed self.q : k_vars_random self.N : nobs (total) Notes ----- Fit returns a result instance, but not all results that use the inherited methods have been checked. Parameters need to change: drop formula and we require a naming convention for the units (currently Y,X,Z). - endog, exog_fe, endog_re ? logL does not include constant, e.g. sqrt(pi) llf is for MLE not for REML convergence criteria for iteration Currently convergence in the iterative solver is reached if either the loglikelihood *or* the fixed effects parameter do not change above tolerance. In some examples, the fixed effects parameters converged to 1e-5 within 150 iterations while the log likelihood did not converge within 2000 iterations. This might be the case if the fixed effects parameters are well estimated, but there are still changes in the random effects. If params_rtol and params_atol are set at a higher level, then the random effects might not be estimated to a very high precision. The above was with a misspecified model, without a constant. With a correctly specified model convergence is fast, within a few iterations (6 in example). cX||_t|j|_|j|_t d|jD|_|j|_|jdj}|jd|_ |j |_ tj |j tj |_|jdj}|jd|_|j|_tj |jfdztj |_d|_tj|_dS)Nc3:K|]}|jjdVdS)rN)r r .0units r z'OneWayMixed.__init__..s*<<TV\!_<<<<<z*OneWayMixed._compute_a..$<<<""$$<<.rirN) rWr2r1rrr<rpinvr&rr)r rTrrs r _compute_azOneWayMixed._compute_asJ 1 1D HHTVTVTZ 0 0 0 0 <<<<< = = <<<<< = =F1II  1%%rFcd}|jD]}|r|j}n!||j|j}|jt j|j|j z }|t j |d z }||j dzt j t j|j|j dz|zz zz }t j||jz |_ dS)a, Estimate sigma. If ML is True, return the ML estimate of sigma, else return the REML estimate. If ML, this is (3.6) in Laird, Lange, Stram (see help(Mixed)), otherwise it corresponds to (3.8). sigma is the standard deviation of the noise (residual) rN)rWr!r(r&r%r*rrr r.powerr<rtracerr sqrtr[)r r>sigmasqrTr!r's r_compute_sigmazOneWayMixed._compute_sigmasJ B BD Fty)))F///A rx1~~))++ +G tz1}rx DF0C0C/3z1}q/@1A(B(BB BGGWWtv-.. rc d}|jD]}|r|j}n!||j|j}|t j|j|jz }t j |j |j }||j t j t j |j ||z z }||j z |_ dS)a  Estimate random effects covariance D. If ML is True, return the ML estimate of sigma, else return the REML estimate. If ML, this is (3.7) in Laird, Lange, Stram (see help(Mixed)), otherwise it corresponds to (3.9). rnN)rWr!r(r&r%rmultiplyouterr.rr rrrY)r r>rrTr!r's r _compute_DzOneWayMixed._compute_D5s J 4 4D Fty)))F ""464622 2Atvtv&&A "&Q333 3AATVrc|jS)z Approximate covariance of estimates of fixed effects. Just after Display (3.10) in Laird, Lange, Stram (see help(Mixed)). )r&r"s r cov_fixedzOneWayMixed.cov_fixedKs yrc|jS)z Estimate random effects covariance D. If ML is True, return the ML estimate of sigma, else return the REML estimate. see _compute_D, alias for self.D )rr"s rr8zOneWayMixed.cov_randomUs v rc|jS)z| estimated coefficients for exogeneous variables or fixed effects see _compute_a, alias for self.a )r1r"s rparamszOneWayMixed.params_s v rcHtjd|jDS)z+random coefficients for each unit cg|] }|j SrM)r.rRs rrhz3OneWayMixed.params_random_units..ms777D777r)rarrayrWr"s rparams_random_unitszOneWayMixed.params_random_unitshs% x77DJ777888rc*|S)z estimated covariance for coefficients for exogeneous variables or fixed effects see cov_fixed, and Sinv in _compute_a )ryr"s r cov_paramszOneWayMixed.cov_paramsos ~~rcrtjtj|S)z] standard errors of estimated coefficients for exogeneous variables (fixed) )rrqdiagrr"s rbsezOneWayMixed.bsexs( wrwt0011222rc4d||zSrArDrEs rrFzOneWayMixed.deviancesDIII$$$$rcd}|jD]!}|||j|z }"|s1|tjt j|jdz z }|S)z9 Return log-likelihood, REML by default. rn)r1r>r)rWr?r1rr:rr;r&)r r>r?rTs rr?zOneWayMixed.logLsj J / /D DII2I.. .DD 1 BF15++,,q0 0D rcntd|jD}td|jD}tj||dd|_d}d}d}|jD]}|jt j|j|jz |_ |j dkr-tj|j |j dd|_ nS|j |j jddf}tj||j dd|_ |t j|jd|jt j|jj|jzz |j t j|j j|j zz z }|t j|j |j z }|tjt j|j j|j z }|j|jdz |j zz |jz |_||j|jdz |j zz |jz z}t j||_|||zz |jz |_dS)NcVg|]&}tj|jj|j'SrM)rrr rrRs rrhz*OneWayMixed.initialize..*AAAd$&))AAArcVg|]&}tj|jj|j'SrM)rrr rrrRs rrhz*OneWayMixed.initialize..rr)rcondrrVr)r<rWrlstsqr1rrrr r*rar r.reshaper rorrurvrkr[rYr]df_residrqrr)r rrrr'rrrTr s r initializezOneWayMixed.initializesG AAdjAAA B B AAdjAAA B BAR(((+  J 2 2DVbfTVTV444DFvzzr:::1=FNNDFLOQ#788DF"555a8 ++//11"&46":"::??AAB"&46":"::??AAB CG ""464622 2A tvx0011 1AA46A:"77$&@ DFdfqjDF22TV;<WW%% gk/TV+rh㈵>-C6?c|||jc|_}|jd|j|jd|j|jd|jtj|j|z |jz |kr d|_ dStj tj |j|j z ||jz|zkr d|_ dS|j|_ dS)z4convergence check for iterative estimation rCllfr|rFT) rFrdhistoryappendr1copyrrfabs terminationallabs_a_old)r r>rtol params_rtol params_atololds rcontzOneWayMixed.conts   ,,dh # U""48,,, X%%dfkkmm444 S  /// 7DHsNdh. / /$ 6 6$D 5 6"&$+-..+2F2TU V V 'D 5v{{}} trdư>ctj|jz|_gggd|_t |D]]}|||||| ||||sn^d|_ td||_ t|}d|_||_|S)N)rr|rrC)r>rrrmaxiterz-Warning: maximum number of iterations reachedrV)rrcr1rrrangerlrsrwrrprint iterationsOneWayMixedResultsscalernormalized_cov_params)r rr>rrriresultss rr2zOneWayMixed.fitsftvo  2266 w C CA OO      2  & & & OOrO " " "99;9DFF   )D  A B B B$T** (,(9(9%rNrH)Frrr)rFrrr)rIrJrKrLrrlrsrwryr8propertyr|rrrrFr?rrr2rMrrrOrOsDDDL0&&& ////.,X99X9    33X3%%%%    ,,,84rrOcpeZdZdZdZedZedZdZ ddZ dZ dd Z dd Z dd Zd S)rz*Results class for OneWayMixed models c,||_|j|_dSrG)modelr|)r rs rrzOneWayMixedResults.__init__s l rc8|jdS)NTrC)rr?r"s rrzOneWayMixedResults.llfsz$'''rc|jjSrG)rrr"s rrz&OneWayMixedResults.params_random_unitss z--rc4|jSrG)rr8r"s rr8zOneWayMixedResults.cov_randomsz$$&&&rlastexogc"|dkr|j|jj d}nmt|tr:t ||jjkst d|j|}ntj|jj}|S)Nrz&length of idx different from k_exog_re) r|rrb isinstancelistrXr=rr_)r idxmeanrs r mean_randomzOneWayMixedResults.mean_randoms *  K!5 5 6 67EE T " " 3s88tz333 !IJJJ C(HTZ122E rcrtjtj|SrG)rrqrr8r"s r std_randomzOneWayMixedResults.std_random s&wrwt0011222rNTc ddlm}ddlm}|}|jj}|dkr'ttj |dzd}}n|d}}|dd|jj d zzz}|r| } ndg|z} | } t|D]} |||| } | | | |jdd| fz|d \} }}tj|d|d d }| d| z| |||| | | | d|S)acreate plot of marginal distribution of random effects Parameters ---------- bins : int or bin edges option for bins in matplotlibs hist method. Current default is not very sophisticated. All distributions use the same setting for bins. use_loc : bool If True, then the distribution with mean given by the fixed effect is used. Returns ------- Figure figure with subplots Notes ----- What can make this fancier? Bin edges will not make sense if loc or scale differ across random effect distributions. rN)normg?rrVgUUUUUU?T)binsnormedrz&Random Effect %d Marginal Distribution)locrr*)matplotlib.pyplotpyplot scipy.statsrfigurerrbintrceilrZrrr add_subplothistrlinspace set_titleplotpdf)r ruse_locpltnormalfigkrowscolsrriiaxfreqbins__pointss rplot_random_univariatez)OneWayMixedResults.plot_random_univariates6 ('''''......jjll J  q55RWQW--..$DDA$D <q4:-666D  ""$$CC#a%C!!((  BtR00BWWSWt/G"/M%M)-d%<>>>@''''R777777rr) rLnumpyr numpy.linalglinalgrstatsmodels.base.modelrstatsmodels.tools.decoratorsrrrOrrMrrrs&999999777777R&R&R&R&R&R&R&R&juuuuuuuup X7X7X7X7X7/X7X7X7X7X7r