M/PhddZddlmZddlmZddlmZmZddlm Z ddl m Z ddl Z ddl ZddlmZmZdd lmZmZdd lmZddlmcmZddlmcmZdd lmZdd lm Z dd l!m"Z"m#Z#ddl$m%Z%m&Z&ddl'm(Z(ddl)m*Z*ddl+m,Z,m-Z-m.Z.ddl/m0Z1dZ2gdZ3dZ4dZ5Gddej6Z7Gdde7Z8Gdde7Z9Gdde9Z:Gd d!e8Z; d-d%ZZ?Gd)d*e?Z@Gd+d,ejAZBejCeBe?dS)/aH This module implements standard regression models: Generalized Least Squares (GLS) Ordinary Least Squares (OLS) Weighted Least Squares (WLS) Generalized Least Squares with autoregressive error terms GLSAR(p) Models are specified with an endogenous response variable and an exogenous design matrix and are fit using their `fit` method. Subclasses that have more complicated covariance matrices should write over the 'whiten' method as the fit method prewhitens the response by calling 'whiten'. General reference for regression models: D. C. Montgomery and E.A. Peck. "Introduction to Linear Regression Analysis." 2nd. Ed., Wiley, 1992. Econometrics references for regression models: R. Davidson and J.G. MacKinnon. "Econometric Theory and Methods," Oxford, 2004. W. Green. "Econometric Analysis," 5th ed., Pearson, 2003. ) annotations)Appender)lrangelzip)Literal)SequenceN)optimizestats)choleskytoeplitz)dtrtri) _ELRegOpts)PredictionResults)cache_readonlycache_writable)InvalidTestWarning ValueWarning) pinv_extended) Float64Array) bool_like float_like string_like) _predictionzrestructuredtext en)GLSWLSOLSGLSARrRegressionResultsWrappera, Return a regularized fit to a linear regression model. Parameters ---------- method : str Either 'elastic_net' or 'sqrt_lasso'. alpha : scalar or array_like The penalty weight. If a scalar, the same penalty weight applies to all variables in the model. If a vector, it must have the same length as `params`, and contains a penalty weight for each coefficient. L1_wt : scalar The fraction of the penalty given to the L1 penalty term. Must be between 0 and 1 (inclusive). If 0, the fit is a ridge fit, if 1 it is a lasso fit. start_params : array_like Starting values for ``params``. profile_scale : bool If True the penalized fit is computed using the profile (concentrated) log-likelihood for the Gaussian model. Otherwise the fit uses the residual sum of squares. refit : bool If True, the model is refit using only the variables that have non-zero coefficients in the regularized fit. The refitted model is not regularized. **kwargs Additional keyword arguments that contain information used when constructing a model using the formula interface. Returns ------- statsmodels.base.elastic_net.RegularizedResults The regularized results. Notes ----- The elastic net uses a combination of L1 and L2 penalties. The implementation closely follows the glmnet package in R. The function that is minimized is: .. math:: 0.5*RSS/n + alpha*((1-L1\_wt)*|params|_2^2/2 + L1\_wt*|params|_1) where RSS is the usual regression sum of squares, n is the sample size, and :math:`|*|_1` and :math:`|*|_2` are the L1 and L2 norms. For WLS and GLS, the RSS is calculated using the whitened endog and exog data. Post-estimation results are based on the same data used to select variables, hence may be subject to overfitting biases. The elastic_net method uses the following keyword arguments: maxiter : int Maximum number of iterations cnvrg_tol : float Convergence threshold for line searches zero_tol : float Coefficients below this threshold are treated as zero. The square root lasso approach is a variation of the Lasso that is largely self-tuning (the optimal tuning parameter does not depend on the standard deviation of the regression errors). If the errors are Gaussian, the tuning parameter can be taken to be alpha = 1.1 * np.sqrt(n) * norm.ppf(1 - 0.05 / (2 * p)) where n is the sample size and p is the number of predictors. The square root lasso uses the following keyword arguments: zero_tol : float Coefficients below this threshold are treated as zero. The cvxopt module is required to estimate model using the square root lasso. References ---------- .. [*] Friedman, Hastie, Tibshirani (2008). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software 33(1), 1-22 Feb 2010. .. [*] A Belloni, V Chernozhukov, L Wang (2011). Square-root Lasso: pivotal recovery of sparse signals via conic programming. Biometrika 98(4), 791-806. https://arxiv.org/pdf/1009.5689.pdf c8|dStj|}|jdkrtj||}|jdkr<|j|fkrt d|d|d|dtj|z }n|j||fkrt d|d|d|tt|d dd \}}|dkrtj d |dkrt d |z||fS) z Returns sigma (matrix, nobs by nobs) for GLS and the inverse of its Cholesky decomposition. Handles dimensions and checks integrity. If sigma is None, returns None, None. Otherwise returns sigma, cholsigmainv. NNNrrz%Sigma must be a scalar, 1d of length z or a 2d array of shape z x T)lower)r" overwrite_cz8Cholesky decomposition of sigma yields a singular matrixz#Invalid input to dtrtri (info = %d)) npasarraysqueezendimrepeatshape ValueErrorsqrtr r linalg LinAlgError)sigmanobs cholsigmainvinfos c/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/regression/linear_model.py _get_sigmar3sd }z Ju   % % ' 'E zQ %&& zQ ;4' ! !*9=tttTTKLL L' ;4, & &*9=tttTTKLL L#HU$$?$?$?*.DBBB d !88)'')CDD D AXXBTIJJ J , ceZdZdZfdZdZedZejdZedZ e jdZ dZ dddZ ddZ ddZ xZS)RegressionModelzp Base class for linear regression models. Should not be directly called. Intended for subclassing. c tj||fi|d|_|jgddS)N) pinv_wexogwendogwexogweights)super__init__r8 _data_attrextend)selfendogexogkwargs __class__s r2r=zRegressionModel.__init__sN//////3 KKKLLLLLr4c||j|_||j|_t |jjd|_d|_d|_ d|_ dS)zInitialize model components.rN) whitenrBr:rAr9floatr)r/ _df_model _df_residrankr@s r2 initializezRegressionModel.initializes^[[++ kk$*-- $**1-..  r4c|jQ|j)tj|j|_t |j|jz |_|jS)z The model degree of freedom. The dof is defined as the rank of the regressor matrix minus 1 if a constant is included. )rHrJr$r, matrix_rankrBrG k_constantrKs r2df_modelzRegressionModel.df_modelsM > !y I11$)<< "49t#>??DN~r4c||_dSN)rHr@values r2rPzRegressionModel.df_model r4c|jD|j)tj|j|_|j|jz |_|jS)z The residual degree of freedom. The dof is defined as the number of observations minus the rank of the regressor matrix. )rIrJr$r,rNrBr/rKs r2df_residzRegressionModel.df_residsE > !y I11$)<< !Y2DN~r4c||_dSrR)rIrSs r2rWzRegressionModel.df_residrUr4c td)z Whiten method that must be overwritten by individual models. Parameters ---------- x : array_like Data to be whitened. zSubclasses must implement.)NotImplementedErrorr@xs r2rFzRegressionModel.whitens"">???r4pinv nonrobustNmethodLiteral['pinv', 'qr']cov_typenLiteral['nonrobust', 'fixed scale', 'HC0', 'HC1', 'HC2', 'HC3', 'HAC', 'hac-panel', 'hac-groupsum', 'cluster']use_t bool | Nonec $|dkrt|dr t|drt|dst|j\|_}t j|jt j|j|_||_tj t j ||_ t j|j|j }n|dkr{t|dr0t|dr t|drt|dstj |j\}} || c|_|_tj t j| j| |_tj | dd|_tj | |_ n|j|j} }tj |j|_t j|j|j x|_} tj | | }nt/d |j!t3|j |jz |_|j|j|j z |_t=|t>rtA|||j||| } ntC||f|j|||d |} tE| S) a Full fit of the model. The results include an estimate of covariance matrix, (whitened) residuals and an estimate of scale. Parameters ---------- method : str, optional Can be "pinv", "qr". "pinv" uses the Moore-Penrose pseudoinverse to solve the least squares problem. "qr" uses the QR factorization. cov_type : str, optional See `regression.linear_model.RegressionResults` for a description of the available covariance estimators. cov_kwds : list or None, optional See `linear_model.RegressionResults.get_robustcov_results` for a description required keywords for alternative covariance estimators. use_t : bool, optional Flag indicating to use the Student's t distribution when computing p-values. Default behavior depends on cov_type. See `linear_model.RegressionResults.get_robustcov_results` for implementation details. **kwargs Additional keyword arguments that contain information used when constructing a model using the formula interface. Returns ------- RegressionResults The model estimation results. See Also -------- RegressionResults The results container. RegressionResults.get_robustcov_results A method to change the covariance estimator used when fitting the model. Notes ----- The fit method uses the pseudoinverse of the design/exogenous variables to solve the least squares minimization. r]r8normalized_cov_paramsrJqrexog_Qexog_Rrzmethod has to be "pinv" or "qr"N)rfracov_kwdsrc)#hasattrrr:r8r$dot transposerfwexog_singular_valuesr,rNdiagrJr9rgrhriinvTsvdr]effectssolver*rHrGrOrIr/rW isinstancer OLSResultsRegressionResultsr) r@r_rarjrcrCsingular_valuesbetaQRrslfits r2fitzRegressionModel.fitsB V  D,// LD"9:: LD&)) L4A3L3L0-/VOR\$/%B%B.D.D*.=*I11"'/2J2JKK 6$/4;77DD t^^D(++ 0D(++ 0D"9:: 0D&)) 0y||DJ//1+,a( T[-/Y]]26!#q>>-J-J*.0Y]]1a-C-C*I11!44 {DK1 innTZ88DO%'VAC%=%= =DL79??1g..DD>?? ? > !"49t#>??DN > ! I 1DM dC d&*&@!HECCCDD %d&*&@!HE D (---r4c>||j}tj||S)a Return linear predicted values from a design matrix. Parameters ---------- params : array_like Parameters of a linear model. exog : array_like, optional Design / exogenous data. Model exog is used if None. Returns ------- array_like An array of fitted values. Notes ----- If the model has not yet been fit, params is not optional. )rBr$rl)r@paramsrBs r2predictzRegressionModel.predicts". <9DvdF###r4c|||}|ddlm}|}||tj|}|S)a Construct a random number generator for the predictive distribution. Parameters ---------- params : array_like The model parameters (regression coefficients). scale : scalar The variance parameter. exog : array_like The predictor variable matrix. dist_class : class A random number generator class. Must take 'loc' and 'scale' as arguments and return a random number generator implementing an ``rvs`` method for simulating random values. Defaults to normal. Returns ------- gen Frozen random number generator object with mean and variance determined by the fitted linear model. Use the ``rvs`` method to generate random values. Notes ----- Due to the behavior of ``scipy.stats.distributions objects``, the returned random number generator must be called with ``gen.rvs(n)`` where ``n`` is the number of observations in the data set used to fit the model. If any other value is used for ``n``, misleading results will be produced. Nr)norm)locscale)rscipy.stats.distributionsrr$r+)r@rrrB dist_classr}rgens r2get_distributionz RegressionModel.get_distributionsV@ll64((   6 6 6 6 6 6JjS777 r4)r]r^NN)r_r`rarbrcrdrRr!)__name__ __module__ __qualname____doc__r=rLpropertyrPsetterrWrFr}rr __classcell__rDs@r2r6r6s/ MMMMM     X __  X __ @ @ @-3!%!v.v.v.v.v.p$$$$8%%%%%%%%r4r6ceZdZdejejejzZdfd Z dZ dZ dd Z e e ddZxZS)ra Generalized Least Squares {params} sigma : scalar or array The array or scalar `sigma` is the weighting matrix of the covariance. The default is None for no scaling. If `sigma` is a scalar, it is assumed that `sigma` is an n x n diagonal matrix with the given scalar, `sigma` as the value of each diagonal element. If `sigma` is an n-length vector, then `sigma` is assumed to be a diagonal matrix with the given `sigma` on the diagonal. This should be the same as WLS. {extra_params} Attributes ---------- pinv_wexog : ndarray `pinv_wexog` is the p x n Moore-Penrose pseudoinverse of `wexog`. cholsimgainv : ndarray The transpose of the Cholesky decomposition of the pseudoinverse. df_model : float p - 1, where p is the number of regressors including the intercept. of freedom. df_resid : float Number of observations n less the number of parameters p. llf : float The value of the likelihood function of the fitted model. nobs : float The number of observations n. normalized_cov_params : ndarray p x p array :math:`(X^{{T}}\Sigma^{{-1}}X)^{{-1}}` results : RegressionResults instance A property that returns the RegressionResults class if fit. sigma : ndarray `sigma` is the n x n covariance structure of the error terms. wexog : ndarray Design matrix whitened by `cholsigmainv` wendog : ndarray Response variable whitened by `cholsigmainv` See Also -------- WLS : Fit a linear model using Weighted Least Squares. OLS : Fit a linear model using Ordinary Least Squares. Notes ----- If sigma is a function of the data making one of the regressors a constant, then the current postestimation statistics will not be correct. Examples -------- >>> import statsmodels.api as sm >>> data = sm.datasets.longley.load() >>> data.exog = sm.add_constant(data.exog) >>> ols_resid = sm.OLS(data.endog, data.exog).fit().resid >>> res_fit = sm.OLS(ols_resid[1:], ols_resid[:-1]).fit() >>> rho = res_fit.params `rho` is a consistent estimator of the correlation of the residuals from an OLS fit of the longley data. It is assumed that this is the true rho of the AR process data. >>> from scipy.linalg import toeplitz >>> order = toeplitz(np.arange(16)) >>> sigma = rho**order `sigma` is an n x n matrix of the autocorrelation structure of the data. >>> gls_model = sm.GLS(data.endog, data.exog, sigma=sigma) >>> gls_results = gls_model.fit() >>> print(gls_results.summary()) r extra_paramsNnonec t|tur||t|t |\}}t j||f||||d||jddgdS)N)missinghasconstr.r0r.r0) typer _check_kwargsr3lenr<r=r>r?) r@rArBr.rrrCr0rDs r2r=z GLS.__init__s ::     v & & &)E ;;| Gg+35/; G G?E G G G 899999r4ctj|}|j|jjdkr|S|jjdkr)|jdkr ||jzS||jdddfzStj|j|S)aG GLS whiten method. Parameters ---------- x : array_like Data to be whitened. Returns ------- ndarray The value np.dot(cholsigmainv,X). See Also -------- GLS : Fit a linear model using Generalized Least Squares. Nr)r$r%r.r)r'r0rlr[s r2rFz GLS.whitens$ JqMM : !1R!7!7H Z_ ! !v{{4,,,4,QQQW5556$+Q// /r4c*|jdz }tj|jtj|j|z dzd}tj| |z}|dtjtj|z z|zz}tj|j rt|j j dkr3tj |j }|d|dzz}n1|dtjtj|j zz}|S)aU Compute the value of the Gaussian log-likelihood function at params. Given the whitened design matrix, the log-likelihood is evaluated at the parameter vector `params` for the dependent variable `endog`. Parameters ---------- params : array_like The model parameters. Returns ------- float The value of the log-likelihood function for a GLS Model. Notes ----- The log-likelihood function for the normal distribution is .. math:: -\frac{n}{2}\log\left(\left(Y-\hat{Y}\right)^{\prime} \left(Y-\hat{Y}\right)\right) -\frac{n}{2}\left(1+\log\left(\frac{2\pi}{n}\right)\right) -\frac{1}{2}\log\left(\left|\Sigma\right|\right) Y and Y-hat are whitened. @raxisr?) r/r$sumr9rlr:logpianyr.r'r,slogdet)r@rnobs2SSRllfdets r2loglikez GLS.loglike<s: CfdkBF4:v$>$>>BKKKvc{{lU" "&u%%%u,, 6$*   6z!##i'' 33r#a&y s26"&"4"45555 r4Tc|j|jjdkr$tj|jjdS|jjdkr|jStj|jS)a Compute weights for calculating Hessian. Parameters ---------- params : ndarray The parameter at which Hessian is evaluated. scale : None or float If scale is None, then the default scale will be calculated. Default scale is defined by `self.scaletype` and set in fit. If scale is not None, then it is used as a fixed scale. observed : bool If True, then the observed Hessian is returned. If false then the expected information matrix is returned. Returns ------- ndarray A 1d weight vector used in the calculation of the Hessian. The hessian is obtained by `(exog.T * hessian_factor).dot(exog)`. Nrrr)r.r)r$onesrBr'r0ror@rrobserveds r2hessian_factorzGLS.hessian_factorgs_. : !1R!7!7749?1-.. . Z_ ! !$ $74,-- -r4 elastic_net?Fc tj|stj|}|j|jjdkrtj|j}n'|jjdkr|j}nt d|tjd|z zt|j z }t|j |j j d||||||d|} ddlm} m} | || j} | | S)Nrrzsigma should be 1-dim or 2-dimr_alphaL1_wt start_params profile_scalerefitrRegularizedResultsRegularizedResultsWrapperr)r$isscalarr%r.r'ror*rrrArr9r:fit_regularizedstatsmodels.base.elastic_netrrr) r@r_rrrrrrCvar_obsrsltrrrrslts r2rzGLS.fit_regularizeds3{5!! &Ju%%E : !z!##'$*--A%%* !ABBBBF1w;///#dj//AE;s4; ++;#%' ## " ##        #"455((///r4NrNNTrrrNFFrrrformatbase_model_params_doc_missing_param_doc_extra_param_docrr=rFrrr_fit_regularized_docrrrs@r2rrsIR d,/$2GG  I IS X : : : : : :000:)))V....<X"##:>> import statsmodels.api as sm >>> Y = [1,3,4,5,2,3,4] >>> X = range(1,8) >>> X = sm.add_constant(X) >>> wls_model = sm.WLS(Y,X, weights=list(range(1,8))) >>> results = wls_model.fit() >>> results.params array([ 2.91666667, 0.0952381 ]) >>> results.tvalues array([ 2.0652652 , 0.35684428]) >>> print(results.t_test([1, 0])) >>> print(results.f_test([0, 1])) rrrNc t|tur||tj|}|jdkr]|dkr5d|vr1|d)tj|t|d}n"tj|t|}t|dkr(tj|g}n|}tj ||f|||d||j jd}|j }|j |kr |jd|krtddSdS)Nrdrop missing_idxr)rr;rrz/Weights must be scalar or same length as design)rrrr$arrayr)r(rr&r<r=rBr;sizer*) r@rArBr;rrrCr/rDs r2r=z WLS.__init__sg ::     v & & &(7## =B  6!!mv&=&==)5)GS 1F-G-GHH)GSZZ88 w<<1  h 1 1233GGoo''G Pg*1H P PHN P P Pyq!, <4  GM!$4$<$<NOO O $<$$>>BKKKvc{{lU" "&u%%%u,, sRVBF4<001111 r4Tc|jS)a Compute the weights for calculating the Hessian. Parameters ---------- params : ndarray The parameter at which Hessian is evaluated. scale : None or float If scale is None, then the default scale will be calculated. Default scale is defined by `self.scaletype` and set in fit. If scale is not None, then it is used as a fixed scale. observed : bool If True, then the observed Hessian is returned. If false then the expected information matrix is returned. Returns ------- ndarray A 1d weight vector used in the calculation of the Hessian. The hessian is obtained by `(exog.T * hessian_factor).dot(exog)`. r;rs r2rzWLS.hessian_factor)s .|r4rrFc Ttj|stj|}|tj|jzt |jz }t |j|jj d||||||d|}ddl m } m } | ||j } | | S)Nrrrr)r$rr%rr;rrr9r:rrrrr) r@r_rrrrrrCrrrrs r2rzWLS.fit_regularizedBs {5!! &Ju%%Et|,,,s4>> import statsmodels.api as sm >>> import numpy as np >>> duncan_prestige = sm.datasets.get_rdataset("Duncan", "carData") >>> Y = duncan_prestige.data['income'] >>> X = duncan_prestige.data['education'] >>> X = sm.add_constant(X) >>> model = sm.OLS(Y,X) >>> results = model.fit() >>> results.params const 10.603498 education 0.594859 dtype: float64 >>> results.tvalues const 2.039813 education 6.892802 dtype: float64 >>> print(results.t_test([1, 0])) Test for Constraints ============================================================================== coef std err t P>|t| [0.025 0.975] ------------------------------------------------------------------------------ c0 10.6035 5.198 2.040 0.048 0.120 21.087 ============================================================================== >>> print(results.f_test(np.identity(2))) rNrc (d|vrd}tj|ttj||f||d|d|jvr|jdt|tur| |dgdSdS)Nr;zdWeights are not supported in OLS and will be ignoredAn exception will be raised in the next version.rroffset) warningswarnrr<r= _init_keysremoverrr)r@rArBrrrCmsgrDs r2r=z OLS.__init__s   FC M#| , , , ?g+3 ? ?7= ? ? ?  ' ' O " "9 - - - ::     vz 2 2 2 2 2  r4c|jdz }t|j}|jtj|j|z }t |dr ||jz}tj|dz}|D| tj dtj zz|tj ||z zz |z }n1| tj dtj z|zz|d|zz z }|S)a  The likelihood function for the OLS model. Parameters ---------- params : array_like The coefficients with which to estimate the log-likelihood. scale : float or None If None, return the profile (concentrated) log likelihood (profiled over the scale parameter), else return the log-likelihood using the given scale value. Returns ------- float The likelihood function evaluated at params. rrr) r/rGrAr$rlrBrkrrrr)r@rrrr/residssrrs r2rz OLS.loglikes$ CTY RVDIv666 4 " " ! T[ EfUAX =&"%(5d 1C1C+CCeKCC&26!be)e"3444sagFC r4c|S)aO OLS model whitener does nothing. Parameters ---------- x : array_like Data to be whitened. Returns ------- array_like The input array unmodified. See Also -------- OLS : Fit a linear model using Ordinary Least Squares. rr[s r2rFz OLS.whitens $r4cDt|ds|tj|j|}|j |z}|P|jdtj|jj|zz }|tj||z }|j |z|z S| |z S)a Evaluate the score function at a given point. The score corresponds to the profile (concentrated) log-likelihood in which the scale parameter has been profiled out. Parameters ---------- params : array_like The parameter vector at which the score function is computed. scale : float or None If None, return the profile (concentrated) log likelihood (profiled over the scale parameter), else return the log-likelihood using the given scale value. Returns ------- ndarray The score vector. _wexog_xprodNr) rk_setup_score_hessr$rlr_wexog_x_wendog _wendog_xprodrqr/)r@rrxtxbsdrrs r2scorez OLS.scores0t^,, %  " " $ $ $vd'00##d* =$q26$2F2H28,:,:(::C 26&$'' 'CI:#c) )4%< r4c|j}t|dr ||jz }tj||z|_tj|jj|j|_ tj|jj||_ dS)Nr) r9rkrr$rrrlr:rqrr)r@ys r2rzOLS._setup_score_hesssr K 4 " " DKAVAE]]F4:<<<!vdjlA66r4ct|ds|tj|j|}||jdtj|jj|zz }|tj||z }d|jzd|zz}|j|z tj|||dzz z }|j |zdz S|j |z S)a Evaluate the Hessian function at a given point. Parameters ---------- params : array_like The parameter vector at which the Hessian is computed. scale : float or None If None, return the profile (concentrated) log likelihood (profiled over the scale parameter), else return the log-likelihood using the given scale value. Returns ------- ndarray The Hessian matrix. rNr) rkrr$rlrrrrqouterr/)r@rrrrssrphms r2hessianz OLS.hessians&t^,, %  " " $ $ $vd'00 =$q26$2F2H28,:,:(::C 26&$'' 'Cd**QtV3D"S(28D$+?+?#q&+HHBI:?Q& &%%- -r4TcJtj|jjdS)a Calculate the weights for the Hessian. Parameters ---------- params : ndarray The parameter at which Hessian is evaluated. scale : None or float If scale is None, then the default scale will be calculated. Default scale is defined by `self.scaletype` and set in fit. If scale is not None, then it is used as a fixed scale. observed : bool If True, then the observed Hessian is returned. If false then the expected information matrix is returned. Returns ------- ndarray A 1d weight vector used in the calculation of the Hessian. The hessian is obtained by `(exog.T * hessian_factor).dot(exog)`. r)r$rrBr)rs r2rzOLS.hessian_factor%s.wtyq)***r4rrrFc ~|dvrd|z}t|dddd} | ||dkr)maxiter cnvrg_tolzero_tolrrrr)fit_elasticnetrrF) r_rrr loglike_kwds score_kwds hess_kwdsr check_step)r*updaterrr _sqrt_lassor _fit_ridge)r@r_rrrrrrCrdefaultsrrrresultsrrrrs r2rzOLS.fit_regularized>si 6 6 6;fDCS// !!# $&& \ ! !        %%eUHZ4HIIF((v66G,,W55 5?????? A::??5)) )  %LJII#Q> JtCy ! ! & q1uX qsttWb M ^BF6NNX5 6 6$*di2.//3355! Zr  s#ctj|jd\}}}|j}tj|j|j|z}||z}tj|r(|||jzz}||z } tj|| } ntj |}|jtj||dddf|zz} tj | |z} tj | | tj | |} tj|| } ddl m} | || S)a Fit a linear model using ridge regression. Parameters ---------- alpha : scalar or array_like The penalty weight. If a scalar, the same penalty weight applies to all variables in the model. If a vector, it must have the same length as `params`, and contains a penalty weight for each coefficient. Notes ----- Equivalent to fit_regularized with L1_wt = 0 (but implemented more efficiently). rN)r)r$r,rrrBrqrlrArr/r%ro fill_diagonalrtrr)r@rusvtvqs2sdrvtavdrrs r2rzOLS._fit_ridges0$9==A..1b D F13 # #a ' U ;u   "edi''BVFVAv&&FFJu%%E9rvb%4.1*<===D "A  T1 % % % a((AVAq\\FCCCCCC!!$///r4rrRrr)rrrrrrrrrr=rrFrrrrrrrrrrrs@r2rr\s;3f d,/$2GG  I Ig l 3 3 3 3 3 3@($ $ $ $ L777 . . . .D++++2X"##:>> import statsmodels.api as sm >>> X = range(1,8) >>> X = sm.add_constant(X) >>> Y = [1,3,4,5,8,10,9] >>> model = sm.GLSAR(Y, X, rho=2) >>> for i in range(6): ... results = model.fit() ... print("AR coefficients: {{0}}".format(model.rho)) ... rho, sigma = sm.regression.yule_walker(results.resid, ... order=model.order) ... model = sm.GLSAR(Y, X, rho) ... AR coefficients: [ 0. 0.] AR coefficients: [-0.52571491 -0.84496178] AR coefficients: [-0.6104153 -0.86656458] AR coefficients: [-0.60439494 -0.857867 ] AR coefficients: [-0.6048218 -0.85846157] AR coefficients: [-0.60479146 -0.85841922] >>> results.params array([-0.66661205, 1.60850853]) >>> results.tvalues array([ -2.10304127, 21.8047269 ]) >>> print(results.t_test([1, 0])) >>> print(results.f_test(np.identity(2))) Or, equivalently >>> model2 = sm.GLSAR(Y, X, rho=2) >>> res = model2.iterative_fit(maxiter=6) >>> model2.rho array([-0.60479146, -0.85841922]) rNrrc t|ttjfr>t||_tj|jtj|_ntjtj ||_t|jj dvrtd|jj dkr d|j_ |jj d|_|?tj|tj|j ddff|dd|dStj||fd|i|dS) N)rrz*AR parameters must be a scalar or a vectorr)rrrrr)ruintr$integerorderzerosfloat64rhor&r%rr)r*r<r=r)r@rArBr:rrrCrDs r2r=zGLSAR.__init__sF cC, - - +SDJx BJ77DHHz"*S//22DH48>""&00 !MNNNx~##!%*DJ < EGG UBGU[^Q,?$@$@ 207$ 2 2*0 2 2 2 2 2 EGG UD 2 2' 2*0 2 2 2 2 2r4-C6?c \d}d}g|jgd}t|dz D]}t|dr|`||}|d|j|dkr|j}nTtj tj ||jz tj |z } | |krd}nM|j}t|j |j d \|_} |d |j|s,|dkr&t|dr|`||jd d |i|}|dz|_|s5|jd|j|xjdz c_||_|S)a Perform an iterative two-stage procedure to estimate a GLS model. The model is assumed to have AR(p) errors, AR(p) parameters and regression coefficients are estimated iteratively. Parameters ---------- maxiter : int, optional The number of iterations. rtol : float, optional Relative tolerance between estimated coefficients to stop the estimation. Stops if max(abs(last - current) / abs(last)) < rtol. **kwargs Additional keyword arguments passed to `fit`. Returns ------- RegressionResults The results computed using an iterative fit. Fr )rr:rr8rrTN)r7dfr:historyr)r:rrkr8rLr}appendrr$maxr yule_walkerrr7iterr? converged) r@rrtolrCrDr r?rlastdiff_s r2 iterative_fitzGLSAR.iterative_fits.  33w{## , ,At\** $O OO   hhjjG H  $ $W^ 4 4 4Avv~vbfTGN%:;;bfTllJKK$;; $IE~%gm,0J4AAAKDHa EN ! !$( + + + + Wq[[t\** $O OO   $(5575f551u   OH % , ,W^ < < < LLA LL%r4ctj|tj}|}t |jD]4}||dzd|j||d|dz zz ||dzd<5||jdS)aR Whiten a series of columns according to an AR(p) covariance structure. Whitening using this method drops the initial p observations. Parameters ---------- x : array_like The data to be whitened. Returns ------- ndarray The whitened data. rNr)r$r%r9copyrr7r:)r@r\_xr s r2rFz GLSAR.whitenVs" Jq"* % % VVXXtz"" F FAq1uhh<$(1+!a!eH* *EEBAxxLL$*++r4)NrrN)r;r<) rrrrrrrrrr=rIrFrrs@r2rrs3f d,/$2GG  I Ig n2222220????Br4radjustedFTc`t|dd}|dkrtjdtd}|dvrt dt j|t j }|r9|jj st j |d }|| z}|p |j d }|dk}|j d kr |j d d krt dt j|d zt j}|dz|z |d <t!d |d zD]6} |d | || dz|| |zz z || <7t#|dd} t j| |d d} nx#t jj$ra} dt+| vrEtjdt,t j| |d dz} nYd} ~ nd} ~ wwxYw|d |d d| zz } t j| s| d krt j| }n t j}|r"| |t j| fS| |fS)a Estimate AR(p) parameters from a sequence using the Yule-Walker equations. Adjusted or maximum-likelihood estimator (mle) Parameters ---------- x : array_like A 1d array. order : int, optional The order of the autoregressive process. Default is 1. method : str, optional Method can be 'adjusted' or 'mle' and this determines denominator in estimate of autocorrelation function (ACF) at lag k. If 'mle', the denominator is n=X.shape[0], if 'adjusted' the denominator is n-k. The default is adjusted. df : int, optional Specifies the degrees of freedom. If `df` is supplied, then it is assumed the X has `df` degrees of freedom rather than `n`. Default is None. inv : bool If inv is True the inverse of R is also returned. Default is False. demean : bool True, the mean is subtracted from `X` before estimation. Returns ------- rho : ndarray AR(p) coefficients computed using the Yule-Walker method. sigma : float The estimate of the residual standard deviation. See Also -------- burg : Burg's AR estimator. Notes ----- See https://en.wikipedia.org/wiki/Autoregressive_moving_average_model for further details. Examples -------- >>> import statsmodels.api as sm >>> from statsmodels.datasets.sunspots import load >>> data = load() >>> rho, sigma = sm.regression.yule_walker(data.endog, order=4, ... method="mle") >>> rho array([ 1.28310031, -0.45240924, -0.20770299, 0.04794365]) >>> sigma 16.808022730464351 r_)rMunbiasedmle)rrOzunbiased is deprecated in factor of adjusted to reflect that the term is adjusting the sample size used in the autocovariance calculation rather than estimating an unbiased autocovariance. After release 0.13, using 'unbiased' will raise.rM)rMrPz1ACF estimation method must be 'adjusted' or 'MLE')dtypeW) requirementsrrz,expecting a vector to estimate AR parametersrNr zSingular matrixzMatrix is singular. Using pinv.)rrr FutureWarningr*r$rr9flags writeablerequiremeanr)r'r8rrr r,rtr-strrr]isnanr+nanrp)r\r7r_r>rpdemeanr adj_neededr2kr{r:errsigmasqr.s r2rBrBpsz"AF  ?      (((LMMM "*%%%A w  0 13///A QVVXX  agajA:%JvzzagajAooGHHH q"*%%A F<<>>A AaD 1eAg  >>!QB$!ABB%$$&&!a*n*<=!3B3Aiooa122'' 9  C ( ( M;\ J J J)..##ae+CC  CCCCdaeCi__&&&G 8G  1   E29==++++Ezs(F--H"AHH"c^ddlm}m}tjtj|}|jdkrtdt|}|dkrtd|r|| z }||||\}}||\}}||dfS)a Compute Burg's AP(p) parameter estimator. Parameters ---------- endog : array_like The endogenous variable. order : int, optional Order of the AR. Default is 1. demean : bool, optional Flag indicating to subtract the mean from endog before estimation. Returns ------- rho : ndarray The AR(p) coefficients computed using Burg's algorithm. sigma2 : float The estimate of the residual variance. See Also -------- yule_walker : Estimate AR parameters using the Yule-Walker method. Notes ----- AR model estimated includes a constant that is estimated using the sample mean (see [1]_). This value is not reported. References ---------- .. [1] Brockwell, P.J. and Davis, R.A., 2016. Introduction to time series and forecasting. Springer. Examples -------- >>> import statsmodels.api as sm >>> from statsmodels.datasets.sunspots import load >>> data = load() >>> rho, sigma2 = sm.regression.linear_model.burg(data.endog, order=4) >>> rho array([ 1.30934186, -0.48086633, -0.20185982, 0.05501941]) >>> sigma2 271.2467306963966 r)levinson_durbin_pacf pacf_burgrz'endog must be 1-d or squeezable to 1-d.z&order must be an integer larger than 1)r\r ) statsmodels.tsa.stattoolsrbrcr$r&r%r'r*r5rX) rAr7r\rbrcpacfr.arrHs r2burgrgs^JIIIIIII Jrz%(( ) )E zQBCCC JJE qyyABBB % $)E5888KD%  & &EB uRy=r4c~eZdZdZiZ dAfd ZdBfd ZedZed Z ed Z ed Z e d Z ed ZedZedZedZedZedZedZedZedZedZedZedZedZedZdCdZedZedZdZd Z ed!Z!ed"Z"ed#Z#ed$Z$ed%Z%ed&Z&ed'Z'ed(Z(ed)Z)d*Z*dDd-Z+d.Z,dEd/Z-dFd1Z.e/e0j1j dGd2Z1 dHdId<Z2 dJdKd@Z3xZ4S)Lrwa This class summarizes the fit of a linear regression model. It handles the output of contrasts, estimates of covariance, etc. Parameters ---------- model : RegressionModel The regression model instance. params : ndarray The estimated parameters. normalized_cov_params : ndarray The normalized covariance parameters. scale : float The estimated scale of the residuals. cov_type : str The covariance estimator used in the results. cov_kwds : dict Additional keywords used in the covariance specification. use_t : bool Flag indicating to use the Student's t in inference. **kwargs Additional keyword arguments used to initialize the results. Attributes ---------- pinv_wexog See model class docstring for implementation details. cov_type Parameter covariance estimator used for standard errors and t-stats. df_model Model degrees of freedom. The number of regressors `p`. Does not include the constant if one is present. df_resid Residual degrees of freedom. `n - p - 1`, if a constant is present. `n - p` if a constant is not included. het_scale adjusted squared residuals for heteroscedasticity robust standard errors. Is only available after `HC#_se` or `cov_HC#` is called. See HC#_se for more information. history Estimation history for iterative estimators. model A pointer to the model instance that called fit() or results. params The linear coefficients that minimize the least squares criterion. This is usually called Beta for the classical linear model. Nrr^c t||||i|_t|dr |j|_nd|_|j|_|j|_|dkrd|_ddi|_ |d}||_ n2|i}d|vr| d} || }|j d|d|d||D]} t|| || dS) Nrnr^ descriptionzWStandard Errors assume that the covariance matrix of the errors is correctly specified.Trc)rause_selfrcr)r<r=_cacherkrn_wexog_singular_valuesrPrWrarjrcpopget_robustcov_resultssetattr) r@modelrrfrrarjrcrCuse_t_2keyrDs r2r=zRegressionResults.__init__Us>  60% 9 9 9 51 2 2 /*/*ED ' '*.D '  { " "'DM DM}DJJ(""",,w//=#E &D & @4-2 @ @6> @ @ @ , ,C D#vc{ + + + + , ,r4皙?cNt||}|S)a8 Compute the confidence interval of the fitted parameters. Parameters ---------- alpha : float, optional The `alpha` level for the confidence interval. The default `alpha` = .05 returns a 95% confidence interval. cols : array_like, optional Columns to include in returned confidence intervals. Returns ------- array_like The confidence intervals. Notes ----- The confidence interval is based on Student's t-distribution. )rcols)r<conf_int)r@rrvcirDs r2rwzRegressionResults.conf_intzs&,WW  E  5 5 r4cJt|jjjdS)zNumber of observations n.r)rGrqr:r)rKs r2r/zRegressionResults.nobssTZ%+A.///r4cV|j|j|jjS)z:The predicted values for the original (unwhitened) design.)rqrrrBrKs r2 fittedvalueszRegressionResults.fittedvaluess"z!!$+tz???r4cp|jj|j|j|jjz S)zX The residuals of the transformed/whitened regressand and regressor(s). )rqr9rrr:rKs r2wresidzRegressionResults.wresids6 z 4:#5#5 K)$+$++ +r4cp|jj|j|j|jjz S)zThe residuals of the model.)rqrArrrBrKs r2rzRegressionResults.resids5z$*"4"4 K#*#** *r4cJ|j}tj|||jz S)z A scale factor for the covariance matrix. The Default value is ssr/(n-p). Note that the square root of `scale` is often called the standard error of the regression. )r}r$rlrWr@r}s r2rzRegressionResults.scales$vff%% 55r4c:|j}tj||S)z$Sum of squared (whitened) residuals.)r}r$rlrs r2rzRegressionResults.ssrsvff%%%r4cj|j}t|dd}t|dd}|=tj|j|}tj||j|z dzzS|tj|j}||}|j || |z }|j|z }||}tj|dzS|j|j z }tj ||S)z> !"\EL,=,=,?,??N6..99 9r4cD|jj}tj||S)z Uncentered sum of squares. The sum of the squared values of the (whitened) endogenous response variable. )rqr9r$rl)r@r9s r2uncentered_tssz RegressionResults.uncentered_tsss "vff%%%r4cL|jr|j|jz S|j|jz S)z The explained sum of squares. If a constant is present, the centered total sum of squares minus the sum of squared residuals. If there is no constant, the uncentered total sum of squares is used. )rOrrrrKs r2esszRegressionResults.esss. ? 2$tx/ /&1 1r4cX|jrd|j|jz z Sd|j|jz z S)z R-squared of the model. This is defined here as 1 - `ssr`/`centered_tss` if the constant is included in the model and 1 - `ssr`/`uncentered_tss` if the constant is omitted. r)rOrrrrKs r2rsquaredzRegressionResults.rsquareds8 ? 4tx 111 1tx 333 3r4cldtj|j|jz |jd|jz zz S)z Adjusted R-squared. This is defined here as 1 - (`nobs`-1)/`df_resid` * (1-`rsquared`) if a constant is included and 1 - `nobs`/`df_resid` * (1-`rsquared`) if no constant is included. r)r$divider/rOrWrrKs r2 rsquared_adjzRegressionResults.rsquared_adjs9BIdi$/94=II4=(*+ +r4ctj|jdkr$tj|jtjS|j|jz S)z~ Mean squared error the model. The explained sum of squares divided by the model degrees of freedom. r)r$allrP full_likerr[rKs r2 mse_modelzRegressionResults.mse_modelsB 6$-3& ' ' 2<"&11 1x %%r4ctj|jdkr$tj|jtjS|j|jz S)z Mean squared error of the residuals. The sum of squared residuals divided by the residual degrees of freedom. r)r$rrWrrr[rKs r2 mse_residzRegressionResults.mse_residsB 6$-3& ' ' 2<"&11 1x %%r4ctj|j|jzdkr$tj|jtjS|jr|j|j|jzz S|j|j|jzz S)z Total mean squared error. The uncentered total sum of squares divided by the number of observations. r) r$rrWrPrrr[rOrrKs r2 mse_totalzRegressionResults.mse_totalst 6$-$-/36 7 7 ;< 126:: : ? I$  (EF F&$-$-*GH Hr4ct|dr|jdkr|jjd}t j|}|jjj}|jjj dkrQ| tj St|}| |||}|j dkr tj S||}t|j|jd<t|jS|j|jz S)aa F-statistic of the fully specified model. Calculated as the mean squared error of the model divided by the mean squared error of the residuals if the nonrobust covariance is used. Otherwise computed using a Wald-like quadratic form that tests whether all coefficients (excluding the constant) are zero. rar^rrNf_pvalue)rkrarfr)r$eyerqdata const_idxrOr[rrnrf_testrGpvaluerlfvaluerr)r@k_paramsmatridxfts r2rzRegressionResults.fvalue,s 4 $ $ 1+)E)E17:H&""C 1I z)Q..$6MX&& """#h8q==6MS!!B&+BI&6&6DK ### #>$.0 0r4c|jdkr$tj|jtjSt j|j|j|jS)zThe p-value of the F-statistic.r) rPr$rrr[r fsfrWrKs r2rzRegressionResults.f_pvalueQsE =A  < RV44 4wzz$+t}dmDDDr4crtjtj|S)z/The standard errors of the parameter estimates.)r$r+ro cov_paramsrKs r2bsezRegressionResults.bseYs(wrwt0011222r4c,|dS)z Akaike's information criteria. For a model with a constant :math:`-2llf + 2(df\_model + 1)`. For a model without a constant :math:`-2llf + 2(df\_model)`. aic info_criteriarKs r2rzRegressionResults.aic^!!%(((r4c,|dS)z Bayes' information criteria. For a model with a constant :math:`-2llf + \log(n)(df\_model+1)`. For a model without a constant :math:`-2llf + \log(n)(df\_model)`. bicrrKs r2rzRegressionResults.bichrr4rch|}|j|jz|z}|dkrd|jzd|zzS|dkr)d|jzt j|j|zz}|S|dkrddlm}||j|j|S|dkrdd lm }||j|j|Sd S) aReturn an information criterion for the model. Parameters ---------- crit : string One of 'aic', 'bic', 'aicc' or 'hqic'. dk_params : int or float Correction to the number of parameters used in the information criterion. By default, only mean parameters are included, the scale parameter is not included in the parameter count. Use ``dk_params=1`` to include scale in the parameter count. Returns ------- Value of information criterion. References ---------- Burnham KP, Anderson KR (2002). Model Selection and Multimodel Inference; Springer New York. rrrraiccr)rhqic)rN) r"rPrOrr$rr/statsmodels.tools.eval_measuresrr)r@crit dk_paramsrrrrs r2rzRegressionResults.info_criteriars,zz||=4?2Y> 5===1x</ / U]]TX+ty 1 1H <DJ$9$;; = =r4c0tjd|||S)Nz ij,ik,kj->i)r$einsum)r@abs r2_abat_diagonalz RegressionResults._abat_diagonalsy1a000r4cX|jdz|_||j}|S)zJ Heteroscedasticity robust covariance matrix. See HC0_se. r)r} het_scaler)r@cov_HC0s r2rzRegressionResults.cov_HC0s) a**T^,,r4cx|j|jz |jdzz|_||j}|S)zJ Heteroscedasticity robust covariance matrix. See HC1_se. r)r/rWr}rr)r@cov_HC1s r2rzRegressionResults.cov_HC1s7 DM2DKNC**T^,,r4c|jj}|||j}|jdzd|z z |_||j}|S)zJ Heteroscedasticity robust covariance matrix. See HC2_se. rrrqr:rrfr}rr)r@r:hcov_HC2s r2rzRegressionResults.cov_HC2sT      t'A B Ba1-**T^,,r4c|jj}|||j}|jd|z z dz|_||j}|S)zJ Heteroscedasticity robust covariance matrix. See HC3_se. rrr)r@r:rcov_HC3s r2rzRegressionResults.cov_HC3sU      t'A B B+Q/!3**T^,,r4cXtjtj|jS)a} White's (1980) heteroskedasticity robust standard errors. Notes ----- Defined as sqrt(diag(X.T X)^(-1)X.T diag(e_i^(2)) X(X.T X)^(-1) where e_i = resid[i]. When HC0_se or cov_HC0 is called the RegressionResults instance will then have another attribute `het_scale`, which is in this case is just resid**2. )r$r+rorrKs r2HC0_sezRegressionResults.HC0_se wrwt|,,---r4cXtjtj|jS)aT MacKinnon and White's (1985) heteroskedasticity robust standard errors. Notes ----- Defined as sqrt(diag(n/(n-p)*HC_0). When HC1_se or cov_HC1 is called the RegressionResults instance will then have another attribute `het_scale`, which is in this case is n/(n-p)*resid**2. )r$r+rorrKs r2HC1_sezRegressionResults.HC1_ses wrwt|,,---r4cXtjtj|jS)a MacKinnon and White's (1985) heteroskedasticity robust standard errors. Notes ----- Defined as (X.T X)^(-1)X.T diag(e_i^(2)/(1-h_ii)) X(X.T X)^(-1) where h_ii = x_i(X.T X)^(-1)x_i.T When HC2_se or cov_HC2 is called the RegressionResults instance will then have another attribute `het_scale`, which is in this case is resid^(2)/(1-h_ii). )r$r+rorrKs r2HC2_sezRegressionResults.HC2_serr4cXtjtj|jS)a MacKinnon and White's (1985) heteroskedasticity robust standard errors. Notes ----- Defined as (X.T X)^(-1)X.T diag(e_i^(2)/(1-h_ii)^(2)) X(X.T X)^(-1) where h_ii = x_i(X.T X)^(-1)x_i.T. When HC3_se or cov_HC3 is called the RegressionResults instance will then have another attribute `het_scale`, which is in this case is resid^(2)/(1-h_ii)^(2). )r$r+rorrKs r2HC3_sezRegressionResults.HC3_serr4ct|dstdtj|jjj}tj|jd|z|j j zkr!tj dt|jS|jtj|jz S)z Residuals, normalized to have unit variance. Returns ------- array_like The array `wresid` normalized by the sqrt of the scale to have unit variance. rzMethod requires residuals. z5All residuals are 0, cannot compute normed residuals.)rkr*r$finfor}rQepsr+rrqrArXrrRuntimeWarning)r@rs r2 resid_pearsonzRegressionResults.resid_pearson!stW%% ;9:: :ht{())- 74:  cDJ,<,A,A,C,C!C C C MG   ; ;!4!44 4r4c`|jj|jjkrdS|jj}|jj}||krdS|jj}|j}||dddfz}t jt j|ddz}t j|dS)a Parameters ---------- restricted : Result instance The restricted model is assumed to be nested in the current model. The result instance of the restricted model is required to have two attributes, residual sum of squares, `ssr`, residual degrees of freedom, `df_resid`. Returns ------- nested : bool True if nested, otherwise false Notes ----- A most nests another model if the regressors in the smaller model are spanned by the regressors in the larger model and the regressand is identical. FNrr) rqr/rJr:r}r$r+rXallclose)r@ restricted full_rankrestricted_rankrestricted_exog full_wresidscoresscore_l2s r2 _is_nestedzRegressionResults._is_nested:s, :?j.3 3 35JO $*/  ' '5$*0k  ;qqq$w#777276;;q>>Q#67788{8Q'''r4TFcddlm}ddlmcm}||st d|j}|jj }||dddfz}|j } |j } |j } | | z } | d} |r||jdddfz}d}|r"|| ddddfz }t|dd}|dkrCtj |d z}tj|j|| z }|||z}n|d vr'|tj|j|| z }nx|d kr0|jd }||||| z }nB|d kr-|jd}||||}nt d| | |z| jzz}tj|| }||| fS)a@ Use Lagrange Multiplier test to test a set of linear restrictions. Parameters ---------- restricted : Result instance The restricted model is assumed to be nested in the current model. The result instance of the restricted model is required to have two attributes, residual sum of squares, `ssr`, residual degrees of freedom, `df_resid`. demean : bool Flag indicating whether the demean the scores based on the residuals from the restricted model. If True, the covariance of the scores are used and the LM test is identical to the large sample version of the LR test. use_lr : bool A flag indicating whether to estimate the covariance of the model scores using the unrestricted model. Setting the to True improves the power of the test. Returns ------- lm_value : float The test statistic which has a chi2 distributed. p_value : float The p-value of the test statistic. df_diff : int The degrees of freedom of the restriction, i.e. difference in df between models. Notes ----- The LM test examines whether the scores from the restricted model are 0. If the null is true, and the restrictions are valid, then the parameters of the restricted model should be close to the minimum of the sum of squared errors, and so the scores should be close to zero, on average. r)rpNz-Restricted model is not nested by full model.rFrar^rHC0HC1HC2HC3HACmaxlagsclustergroupsz;Only nonrobust, HC, HAC and cluster are currently connected) numpy.linalgrp%statsmodels.stats.sandwich_covariancer sandwich_covariancerr*r}rqr:r/rWrXrr$rlrqrj S_hac_simpleS_crosssectionchi2r)r@rr\use_lrrpswr}r:rrdf_fulldf_restrdf_diffr*rasigma2xpxs_invrrlm_valuep_values r2compare_lm_testz!RegressionResults.compare_lm_testasPN %$$$$$:::::::::z** NLMM M"  4( I-&g% KKQK    T[D11FF  6fkk!nnT111W55F 4[99 { " "WVQY''F&%((1,CC %%EE 5 5 5Cvx001455EE   mI.GC881<==EE  " "]8,FC))&&99::EE344 4E AC(*--'22'))r4c<t|dddk}t|dddk}|s|rtjdt|j}|j}|j}|j}||z }||z |z |z |z} t j| ||} | | |fS)a Use F test to test whether restricted model is correct. Parameters ---------- restricted : Result instance The restricted model is assumed to be nested in the current model. The result instance of the restricted model is required to have two attributes, residual sum of squares, `ssr`, residual degrees of freedom, `df_resid`. Returns ------- f_value : float The test statistic which has an F distribution. p_value : float The p-value of the test statistic. df_diff : int The degrees of freedom of the restriction, i.e. difference in df between models. Notes ----- See mailing list discussion October 17, This test compares the residual sum of squares of the two models. This is not a valid test, if there is unspecified heteroscedasticity or correlation. This method will issue a warning if this is detected but still return the results under the assumption of homoscedasticity and no autocorrelation (sphericity). rar^zQF test for comparison is likely invalid with robust covariance, proceeding anyway) rrrrrrWr rr) r@r has_robust1 has_robust2ssr_full ssr_restrrrrf_valuers r2compare_f_testz RegressionResults.compare_f_testsDdJ << K z:{CC"#   .+ . MA, . . .8N -&g%x'72X=G'**Wgw77((r4c`|r||dSt|dddk}t|dddk}|s|rtjdt|j}|j}|j}|j}||z } d||z z} tj | | } | | | fS)a Likelihood ratio test to test whether restricted model is correct. Parameters ---------- restricted : Result instance The restricted model is assumed to be nested in the current model. The result instance of the restricted model is required to have two attributes, residual sum of squares, `ssr`, residual degrees of freedom, `df_resid`. large_sample : bool Flag indicating whether to use a heteroskedasticity robust version of the LR test, which is a modified LM test. Returns ------- lr_stat : float The likelihood ratio which is chisquare distributed with df_diff degrees of freedom. p_value : float The p-value of the test statistic. df_diff : int The degrees of freedom of the restriction, i.e. difference in df between models. Notes ----- The exact likelihood ratio is valid for homoskedastic data, and is defined as .. math:: D=-2\log\left(\frac{\mathcal{L}_{null}} {\mathcal{L}_{alternative}}\right) where :math:`\mathcal{L}` is the likelihood of the model. With :math:`D` distributed as chisquare with df equal to difference in number of parameters or equivalently difference in residual degrees of freedom. The large sample version of the likelihood ratio is defined as .. math:: D=n s^{\prime}S^{-1}s where :math:`s=n^{-1}\sum_{i=1}^{n} s_{i}` .. math:: s_{i} = x_{i,alternative} \epsilon_{i,null} is the average score of the model evaluated using the residuals from null model and the regressors from the alternative model and :math:`S` is the covariance of the scores, :math:`s_{i}`. The covariance of the scores is estimated using the same estimator as in the alternative model. This test compares the loglikelihood of the two models. This may not be a valid test, if there is unspecified heteroscedasticity or correlation. This method will issue a warning if this is detected but still return the results without taking unspecified heteroscedasticity or correlation into account. This test compares the loglikelihood of the two models. This may not be a valid test, if there is unspecified heteroscedasticity or correlation. This method will issue a warning if this is detected but still return the results without taking unspecified heteroscedasticity or correlation into account. is the average score of the model evaluated using the residuals from null model and the regressors from the alternative model and :math:`S` is the covariance of the scores, :math:`s_{i}`. The covariance of the scores is estimated using the same estimator as in the alternative model. T)rrar^zQLikelihood Ratio test is likely invalid with robust covariance, proceeding anywayr) rrrrrrrWr rr) r@r large_sampler r llf_full llf_restrrrlrdflrstat lr_pvalues r2compare_lr_testz!RegressionResults.compare_lr_tests`  A'' 4'@@ @tZ==L J K 8 8K G   .+ . MA, . . .8N -&7"Y)*JMM&$// y$&&r4rc ddlm}m}ddlmcm}||}d|vr|d|d<d|vr+t|ds|j|d|d<|dd}|r|}n-| |j |j |j |j }||_||j}d |i|_||_d} |d vr|d d} | durd } | |jd <|dvrA|d|jd<|ddx|jd<} | |j z|_n|dvr_|rt)d|||jd<t+|d|z|_n|dkr|d} | |jd<|d|j} | |jd<|dd}||jd<|d| ddg||jd<||| | ||_n|dkr|d}t5|d s%d!|D}t7j|j}|jd"kr|}||jd<|dd }||jd<|jd#krH| r(tAt7j!|x|_"}|#|||$|_n|jd"krt5|d%r|j$}| rotAt7j!|dddf}tAt7j!|ddd#f}||f|_"tK||}|&|||$d|_nt)d&|d|jd<n|d'kr|d(dx|jd(<}|ddx|jd<}|dx|jd<} |dd}||jd<|d|j} | |jd<|gt7j|}t7j'|dd)|d#dkdd#z(}tA|}nx|gt7j|}t7j'|d#d|dd)kdd#z(}tA|}nt)d*tSdg|z||gz}tA|x|_"}|*|| || |+|_|d,|jd<n |d-kr|d(x|jd(<}|dx|jd<} |dd}||jd<|d|j} | |jd<| rJt7j'|d#d|dd)kdd#z}tA|d#zx|_"}|+|| || |+|_|d.|jd<nt)d/| r |d#z |_,|S)0a Create new results instance with robust covariance as default. Parameters ---------- cov_type : str The type of robust sandwich estimator to use. See Notes below. use_t : bool If true, then the t distribution is used for inference. If false, then the normal distribution is used. If `use_t` is None, then an appropriate default is used, which is `True` if the cov_type is nonrobust, and `False` in all other cases. **kwargs Required or optional arguments for robust covariance calculation. See Notes below. Returns ------- RegressionResults This method creates a new results instance with the requested robust covariance as the default covariance of the parameters. Inferential statistics like p-values and hypothesis tests will be based on this covariance matrix. Notes ----- The following covariance types and required or optional arguments are currently available: - 'fixed scale' uses a predefined scale ``scale``: float, optional Argument to set the scale. Default is 1. - 'HC0', 'HC1', 'HC2', 'HC3': heteroscedasticity robust covariance - no keyword arguments - 'HAC': heteroskedasticity-autocorrelation robust covariance ``maxlags`` : integer, required number of lags to use ``kernel`` : {callable, str}, optional kernels currently available kernels are ['bartlett', 'uniform'], default is Bartlett ``use_correction``: bool, optional If true, use small sample correction - 'cluster': clustered covariance estimator ``groups`` : array_like[int], required : Integer-valued index of clusters or groups. ``use_correction``: bool, optional If True the sandwich covariance is calculated with a small sample correction. If False the sandwich covariance is calculated without small sample correction. ``df_correction``: bool, optional If True (default), then the degrees of freedom for the inferential statistics and hypothesis tests, such as pvalues, f_pvalue, conf_int, and t_test and f_test, are based on the number of groups minus one instead of the total number of observations minus the number of explanatory variables. `df_resid` of the results instance is also adjusted. When `use_t` is also True, then pvalues are computed using the Student's t distribution using the corrected values. These may differ substantially from p-values based on the normal is the number of groups is small. If False, then `df_resid` of the results instance is not adjusted. - 'hac-groupsum': Driscoll and Kraay, heteroscedasticity and autocorrelation robust covariance for panel data # TODO: more options needed here ``time`` : array_like, required index of time periods ``maxlags`` : integer, required number of lags to use ``kernel`` : {callable, str}, optional The available kernels are ['bartlett', 'uniform']. The default is Bartlett. ``use_correction`` : {False, 'hac', 'cluster'}, optional If False the the sandwich covariance is calculated without small sample correction. If `use_correction = 'cluster'` (default), then the same small sample correction as in the case of `covtype='cluster'` is used. ``df_correction`` : bool, optional The adjustment to df_resid, see cov_type 'cluster' above - 'hac-panel': heteroscedasticity and autocorrelation robust standard errors in panel data. The data needs to be sorted in this case, the time series for each panel unit or cluster need to be stacked. The membership to a time series of an individual or group can be either specified by group indicators or by increasing time periods. One of ``groups`` or ``time`` is required. # TODO: we need more options here ``groups`` : array_like[int] indicator for groups ``time`` : array_like[int] index of time periods ``maxlags`` : int, required number of lags to use ``kernel`` : {callable, str}, optional Available kernels are ['bartlett', 'uniform'], default is Bartlett ``use_correction`` : {False, 'hac', 'cluster'}, optional If False the sandwich covariance is calculated without small sample correction. ``df_correction`` : bool, optional Adjustment to df_resid, see cov_type 'cluster' above **Reminder**: ``use_correction`` in "hac-groupsum" and "hac-panel" is not bool, needs to be in {False, 'hac', 'cluster'}. .. todo:: Currently there is no check for extra or misspelled keywords, except in the case of cov_type `HCx` r) descriptionsnormalize_cov_typeNkernel weights_funcrkF)rfrrc)r hac-panel hac-groupsum df_correctionT adjust_df)z fixed scale fixed_scaler rjrrrz:heteroscedasticity robust covariance does not use keywordscov_hacruse_correctionrwithoutwith)r correction)nlagsrr#rrr)cZg|](}tjtj|)Sr)r$r&r%).0groups r2 z;RegressionResults.get_robustcov_results.. s,LLLE"*RZ%6%677LLLr4rr)r#valueszonly two groups are supportedrtimer z'either time or groups needs to be given)rr#z HAC-Panelrz HAC-GroupsumzIcov_type not recognized. See docstring for available options and spelling)-statsmodels.base.covtyperrrr rrncallable kernel_dictrDrqrrfrrarcrjgetcov_params_defaultupperr*rr"weights_bartlettrcov_hac_simplerkr$r%rqr'r&runiquen_groups cov_clusterr,mincov_cluster_2groupsnonzerotolistr cov_nw_panelcov_nw_groupsumdf_resid_inference)r@rarcrCrrrrkresrrrrrr#rr7 n_groups0 n_groups1r-ttnobs_groupidxs r2roz'RegressionResults.get_robustcov_resultsW sz NMMMMMMM:::::::::%%h// v  %+ZZ%9%9F> " V # #HVN5K,L,L #%'^F>4J%KF> "::j%00  "CC.. DK&*&@j!""C   =JE'   ? ? ?"JJ==ME)) $- [! 5 5 5*6}*ECL ',2JJw,C,C CCL !E%*S-F%FC " " ^^  != = = : "9:::*6x~~7G7G*HCL '%,T6HNN+E+@+@CL '&(%6%6G,-&7&/&/C " "^^   * *H%F67++ .LLVLLLF++-{a))%+CL "#ZZ(8$??N-;CL) *{aF0329V3D3D/E/EEDMH)+&*8*A*A&&!!68,,+#]F9!$BIfQQQTl$;$; < BJJLLF !z$''jabbD"I!566q9A=EEGGD  !JKKKQC"HbE7l33H'*8}} 4DMH%'__)- &5&&C "+7{*CCL ' ' ^^   / /*0. 8CL 4 17y0A ACL #g#ZZ(8)DDN-;CL) *!::nb6IJJL+7CL ( 7jabbD"I!566q9A=+.r77Q;6 %'%7%7gt,-&8&/&/C "+7~*FCL ' '>?? ?  2%-\C " r4c 0tj|f||||d|S)N)rB transformr; row_labels)predget_prediction)r@rBrGr;rHrCs r2rJz RegressionResults.get_prediction s8" -y'!--%+-- -r4yname str | NonexnameSequence[str] | NonetitlerrGslimboolc ddlm}m}m}t |dd}t |ddd}||j\} } } } ||j\} }|j}|j}t| | | | | |||d  |_ d d d dgfdddddg}t|dr| d|j gf|jrdnd}d|zdzd|jzgfd|zdzd|jzgfdd|jzgfdd|jzgfd d!d|jzgfd"d|jzgfg}|rMgd#gx}}fd$|D}fd%|D}|dgfgt+|t+|z zz}nJd&d'| zgfd(d'|zgfd)d'| zgfd*d'| zgfg}d+d||jzgfd,d| zgfd-d.| zgfd/d.|zgfg}||jjjd1zd2z}dd3lm}|}|||||||4||||||j5|s||||||d4g}|js| d6t|dr | |jd7|jjj d|jjj d8krd9}| ||d d:kr2d;}|dz }||d z}| |n1|d?kr+d@}|dAz }|dBz }|dCz }||z}| ||rDdDtC|D}|"ddE|#||S)Fa) Summarize the Regression Results. Parameters ---------- yname : str, optional Name of endogenous (response) variable. The Default is `y`. xname : list[str], optional Names for the exogenous variables. Default is `var_##` for ## in the number of regressors. Must match the number of parameters in the model. title : str, optional Title for the top table. If not None, then this replaces the default title. alpha : float, optional The significance level for the confidence intervals. slim : bool, optional Flag indicating to produce reduced set or diagnostic information. Default is False. Returns ------- Summary Instance holding the summary tables and text, which can be printed or converted to various output formats. See Also -------- statsmodels.iolib.summary.Summary : A class that holds summary results. r durbin_watson jarque_bera omni_normtestrF)optionalrPT)rWstrictr )jbjbpvskewkurtosisomniomnipvcondno mineigval)Dep. Variable:N)Model:NzMethod:z Least Squares)zDate:N)zTime:N)No. Observations:N)z Df Residuals:N)z Df Model:NraCovariance Type:z (uncentered)z R-squared:z%#8.3fzAdj. R-squared F-statistic:z%#8.4gProb (F-statistic):z%#6.3g)zLog-Likelihood:NzAIC:zBIC:)rarbrcrdz R-squared:zAdj. R-squared:rgrhc(g|]}|dv |Srrr)elemslimlists r2r+z-RegressionResults.summary.. s'IIIT!W5H5H5H5H5Hr4c(g|]}|dv |Srjrrks r2r+z-RegressionResults.summary.. s'KKK$tAw(7J7J7J7J7Jr4Omnibus:z%#6.3fProb(Omnibus):Skew: Kurtosis:Durbin-Watson:Jarque-Bera (JB): Prob(JB):z%#8.3gz Cond. No.N zRegression Results)Summary)gleftgrightrKrMrO)rKrMrrcu[R² is computed without centering (uncentered) since the model does not contain a constant.rjr9The input rank is higher than the number of observations.rz6The smallest eigenvalue is %6.3g. This might indicate zthat there are z5strong multicollinearity problems or that the design zmatrix is singular.z1The condition number is large, %6.3g. This might zindicate that there are z,strong multicollinearity or other numerical z problems.c*g|]\}}d|dzd|S[rz] rr)r texts r2r+z-RegressionResults.summary..% @666 D)Q(($((666r4Notes:)$statsmodels.stats.stattoolsrTrUrVrrr}rrdictdiagnrkr@rarOrrrrrrrrqrDrstatsmodels.iolib.summaryrwadd_table_2colsadd_table_paramsrcrjrBr) enumerateinsert add_extra_txt)r@rKrMrOrrPrTrUrVrYrZr[r\r]r^rr_top_left rsquared_type top_right diagn_left diagn_rightrwsmryetextwstrrms @r2summaryzRegressionResults.summary sBL          5'E:::vdCCC#.;t{#;#; D$$}T[11 f.&Rd#F6$+BK111 -$ 12##/+' 4 $ $ C OO/$-A B B B"oB? !M1C7$-/02&6<$"3346$x$+'=&>?+h.F-GH.x$(234x$(234    ???H(* )JIIIIIIIHKKKK)KKKI!b c(mmc)nn<=>II&4'89+h.?-@A"X_$56&H)<(=>J -% dk(B(BBC0(R-A'(T/):;'(V*;)<= K =J(1C7:NNE 655555wyy T)#(U  D D D d%uE$(J  0 0 0 +  Z ',E') ! + + +   LL5    4 $ $ 7 LL}5 6 6 6 :?  #djo&;A&> > >ND LL    2;  KD & &D K KD ) )D'"+%D LL     d]]FD / /D B BD K D&=D LL     &66$-e$4$4666E LLH % % %   u % % % r4%.4f float_formatrYc ddlm}m}m}||j\} } } } ||j\} }||j}|j}|j}tdd| zfdd|zfdd| zfdd| zfdd|zfd d| zfd d| zfd d |zfg}dd lm }| }| ||||||| |g}|j s|dt|dr ||jd|jjjd|jjjdkrd}|||ddkr!d|dz}||n |dkrd|z}|||r/dt)|D}|dd|D]}|||S)a Experimental summary function to summarize the regression results. Parameters ---------- yname : str The name of the dependent variable (optional). xname : list[str], optional Names for the exogenous variables. Default is `var_##` for ## in the number of regressors. Must match the number of parameters in the model. title : str, optional Title for the top table. If not None, then this replaces the default title. alpha : float The significance level for the confidence intervals. float_format : str The format for floats in parameters summary. Returns ------- Summary Instance holding the summary tables and text, which can be printed or converted to various output formats. See Also -------- statsmodels.iolib.summary2.Summary A class that holds summary results. rrSroz%.3frprqrrrsrtruzCondition No.:z%.0f)summary2)rrrrMrKrOukR² is computed without centering (uncentered) since the model does not contain a constant.rarjrrzr rzThe smallest eigenvalue is %6.3g. This might indicate that there are strong multicollinearity problems or that the design matrix is singular.r{zThe condition number is large, %6.3g. This might indicate that there are strong multicollinearity or other numerical problems.c*g|]\}}d|dzd|Sr}rrs r2r+z.RegressionResults.summary2.. rr4r)rrTrUrVr}rrrstatsmodels.iolibrrwadd_baseadd_dictrOr@rkrjrqrBr)rradd_text)r@rKrMrOrrrTrUrVrYrZr[r\r]r^dwrr_ diagnosticrrrrrlines r2rzRegressionResults.summary2, sN          $/;t{#;#; D$$}T[11 f ]4; ' '.& &4- ( v / ftm $ &8+ , v{ + &2+ . &4- ( v /     /.....!! d%l!e  = = = j!!!  LL4    4 $ $ 7 LL}5 6 6 6 :?  #djo&;A&> > >ND LL    2;  %'.r{3D LL     d]]#$D LL     &66$-e$4$4666E LLH % % %  D MM$     r4)Nrr^NN)rtNrj)TF)F)rN)NTNN)NNNrtF) rKrLrMrNrOrLrrGrPrQ)NNNrtr) rKrLrMrNrOrLrrGrrY)5rrrrrlr=rwrr/r{r}rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrorrIrJrrrrs@r2rwrw s00dFHJ<@#,#,#,#,#,#,J200^0@@^@++^+**^* ^666&&^& ::^:(&&^& 2 2^ 2 4 4^ 4 + +^ +&&^& & &^ & I I^ I"1"1^"1HEE^E33^3))^)))^)#7#7#7#7J & &^ & / /^ / 111^^^^ . .^ . . .^ . . .^ . . .^ .55^50%(%(%(NX*X*X*X*t3)3)3)je'e'e'e'NggggR Xd!)**@D"&---+*-!%*. $ bbbbbL!%*. $ & eeeeeeeeer4rwc<eZdZdZdZ ddZ dd Z dd ZdS)rva Results class for for an OLS model. Parameters ---------- model : RegressionModel The regression model instance. params : ndarray The estimated parameters. normalized_cov_params : ndarray The normalized covariance parameters. scale : float The estimated scale of the residuals. cov_type : str The covariance estimator used in the results. cov_kwds : dict Additional keywords used in the covariance specification. use_t : bool Flag indicating to use the Student's t in inference. **kwargs Additional keyword arguments used to initialize the results. See Also -------- RegressionResults Results store for WLS and GLW models. Notes ----- Most of the methods and attributes are inherited from RegressionResults. The special methods that are only available for OLS are: - get_influence - outlier_test - el_test - conf_int_el c$ddlm}||S)a Calculate influence and outlier measures. Returns ------- OLSInfluence The instance containing methods to calculate the main influence and outlier measures for the OLS regression. See Also -------- statsmodels.stats.outliers_influence.OLSInfluence A class that exposes methods to examine observation influence. r) OLSInfluence)$statsmodels.stats.outliers_influencer)r@rs r2 get_influencezOLSResults.get_influence s' FEEEEE|D!!!r4bonfrtNFc0ddlm}|||||||S)a Test observations for outliers according to method. Parameters ---------- method : str The method to use in the outlier test. Must be one of: - `bonferroni` : one-step correction - `sidak` : one-step correction - `holm-sidak` : - `holm` : - `simes-hochberg` : - `hommel` : - `fdr_bh` : Benjamini/Hochberg - `fdr_by` : Benjamini/Yekutieli See `statsmodels.stats.multitest.multipletests` for details. alpha : float The familywise error rate (FWER). labels : None or array_like If `labels` is not None, then it will be used as index to the returned pandas DataFrame. See also Returns below. order : bool Whether or not to order the results by the absolute value of the studentized residuals. If labels are provided they will also be sorted. cutoff : None or float in [0, 1] If cutoff is not None, then the return only includes observations with multiple testing corrected p-values strictly below the cutoff. The returned array or dataframe can be empty if t. Returns ------- array_like Returns either an ndarray or a DataFrame if labels is not None. Will attempt to get labels from model_results if available. The columns are the Studentized residuals, the unadjusted p-value, and the corrected p-value according to method. Notes ----- The unadjusted p-value is stats.t.sf(abs(resid), df) where df = df_resid - 1. r) outlier_test)labelsr7cutoff)rr)r@r_rrr7rrs r2rzOLSResults.outlier_test s>^ FEEEEE|D&%"'888 8r4rnmrc tj|j}t}t |t |kr|g||jj|jj|jj |jjj d||| } dtj | t |z } |r | | |jfS| | fStj||} ||jj|jj|jj |jjj d|||f} |dkr&t!j|j| dddd| d} |dkr$t!j|j| dd| d} dtj | t |z } |r| | |j|jfS|r | | |jfS| | fS) a Test single or joint hypotheses using Empirical Likelihood. Parameters ---------- b0_vals : 1darray The hypothesized value of the parameter to be tested. param_nums : 1darray The parameter number to be tested. return_weights : bool If true, returns the weights that optimize the likelihood ratio at b0_vals. The default is False. ret_params : bool If true, returns the parameter vector that maximizes the likelihood ratio at b0_vals. Also returns the weights. The default is False. method : str Can either be 'nm' for Nelder-Mead or 'powell' for Powell. The optimization method that optimizes over nuisance parameters. The default is 'nm'. stochastic_exog : bool When True, the exogenous variables are assumed to be stochastic. When the regressors are nonstochastic, moment conditions are placed on the exogenous variables. Confidence intervals for stochastic regressors are at least as large as non-stochastic regressors. The default is True. Returns ------- tuple The p-value and -2 times the log-likelihood ratio for the hypothesized values. Examples -------- >>> import statsmodels.api as sm >>> data = sm.datasets.stackloss.load() >>> endog = data.endog >>> exog = sm.add_constant(data.exog) >>> model = sm.OLS(endog, exog) >>> fitted = model.fit() >>> fitted.params >>> array([-39.91967442, 0.7156402 , 1.29528612, -0.15212252]) >>> fitted.rsquared >>> 0.91357690446068196 >>> # Test that the slope on the first variable is 0 >>> fitted.el_test([0], [1]) >>> (27.248146353888796, 1.7894660442330235e-07) r) param_numsrArBr/nvarrb0_valsstochastic_exogri'r)maxfunr full_outputdispargspowell)rrr)r$rKrrr_opt_nuis_regressrqrArBr/r)r rcdf new_weightsdeleter fmin fmin_powell new_params) r@rrreturn_weights ret_paramsr_rr opt_fun_instllrpvalx0rs r2el_testzOLSResults.el_test sd%%!|| z??c&kk ) )00%j&Z_Z_Z_*1- /1 1 1Cuz~~c3z??;;;D !D,":::Dy Yvz * *DJ,djo !6q!96* T>>- >',e%&T333346C X  &|'Er341,0222235C5:>>#s:777  l6 8OO O  l66 69 r4Tc| tjd|z d |!dd}|!dd} fd}t j||j}t j|j|} || fS)a Compute the confidence interval using Empirical Likelihood. Parameters ---------- param_num : float The parameter for which the confidence interval is desired. sig : float The significance level. Default is 0.05. upper_bound : float The maximum value the upper limit can be. Default is the 99.9% confidence value under OLS assumptions. lower_bound : float The minimum value the lower limit can be. Default is the 99.9% confidence value under OLS assumptions. method : str Can either be 'nm' for Nelder-Mead or 'powell' for Powell. The optimization method that optimizes over nuisance parameters. The default is 'nm'. stochastic_exog : bool When True, the exogenous variables are assumed to be stochastic. When the regressors are nonstochastic, moment conditions are placed on the exogenous variables. Confidence intervals for stochastic regressors are at least as large as non-stochastic regressors. The default is True. Returns ------- lowerl : float The lower bound of the confidence interval. upperl : float The upper bound of the confidence interval. See Also -------- el_test : Test parameters using Empirical Likelihood. Notes ----- This function uses brentq to find the value of beta where test_beta([beta], param_num)[1] is equal to the critical value. The function returns the results of each iteration of brentq at each value of beta. The current function value of the last printed optimization should be the critical value at the desired significance level. For alpha=.05, the value is 3.841459. To ensure optimization terminated successfully, it is suggested to do el_test([lower_limit], [param_num]). If the optimization does not terminate successfully, consider switching optimization algorithms. If optimization is still not successful, try changing the values of start_int_params. If the current function value repeatedly jumps from a number between 0 and the critical value and a very large number (>50), the starting parameters of the interior minimization need to be changed. rNg{Gz?rctj|gtjgdz S)N)r_rr)rr$r)b0r_ param_numr0r@rs r2rz!OLSResults.conf_int_el..f sS<<")0E0E'-0? AAABDFHI Ir4)r rppfrwr brenthr) r@rsig upper_bound lower_boundr_rrlowerlupperlrs `` `` @r2 conf_int_elzOLSResults.conf_int_elY s~Z^^AGQ ' '  --,,Y7:K  --,,Y7:K I I I I I I I I I K!%Y!799DK $:!,..r4)rrtNFN)rrrr)rtNNrT)rrrrrrrrrr4r2rvrv s$$L"""$=A)-18181818fIJ-.WWWWr;?CGO O O O O O r4rvc eZdZdddddddddddd ZejejjeZiZ ejejj e Z dS)rcolumnsrowscov) chisqsresidr;r} bcov_unscaled bcov_scaledrrrr norm_residN) rrr_attrswrap union_dictsrLikelihoodResultsWrapper _wrap_attrs_methods _wrap_methodsrr4r2rr s  F#$"4#@#G#)++KH$D$5C ""MMMr4r)rrMNFT)rT)Dr __future__rstatsmodels.compat.pandasrstatsmodels.compat.pythonrrtypingrcollections.abcrrnumpyr$scipyr r scipy.linalgr r scipy.linalg.lapackr statsmodels.base.modelrrqstatsmodels.base.wrapperwrapperrstatsmodels.emplike.elregressr"statsmodels.regression._predictionrstatsmodels.tools.decoratorsrrstatsmodels.tools.sm_exceptionsrrstatsmodels.tools.toolsrstatsmodels.tools.typingrstatsmodels.tools.validationrrrrerrI __docformat____all__rr3LikelihoodModelr6rrrrrBrgLikelihoodModelResultsrwrvResultsWrapperrpopulate_wrapperrr4r2rs 6#"""""......22222222$$$$$$!!!!!!!!++++++++&&&&&&%%%%%%%%%'''''''''444444@@@@@@GGGGGGGGLLLLLLLL111111111111KKKKKKKKKK!!!!!!%  ' ' ' \ @>d*D`0`0`0`0`0/`0`0`0Fs0s0s0s0s0/s0s0s0lf0f0f0f0f0#f0f0f0R hhhhhChhhV=Booood;;;;|qqqqq3qqqh+T T T T T "T T T n"""""t2"""4.')))))r4