M/PhB@dZddlZGddZGddeZGddeZGd d eZGd d eZGd deZGddeZ GddZ Gdde Z GddZ Gdde Z dS)a A collection of smooth penalty functions. Penalties on vectors take a vector argument and return a scalar penalty. The gradient of the penalty is a vector with the same shape as the input value. Penalties on covariance matrices take two arguments: the matrix and its inverse, both in unpacked (square) form. The returned penalty is a scalar, and the gradient is returned as a vector that contains the gradient with respect to the free elements in the lower triangle of the covariance matrix. All penalties are subtracted from the log-likelihood, so greater penalty values correspond to a greater degree of penalization. The penaties should be smooth so that they can be subtracted from log likelihood functions and optimized using standard methods (i.e. L1 penalties do not belong here). Nc,eZdZdZddZdZdZdZdS) Penaltya, A class for representing a scalar-value penalty. Parameters ---------- weights : array_like A vector of weights that determines the weight of the penalty for each parameter. Notes ----- The class has a member called `alpha` that scales the weights. ?c"||_d|_dSNrweightsalphaselfr s [/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/base/_penalties.py__init__zPenalty.__init__'s  ct)a A penalty function on a vector of parameters. Parameters ---------- params : array_like A vector of parameters. Returns ------- A scalar penaty value; greater values imply greater penalization. NotImplementedErrorr paramss r funcz Penalty.func+ "!rct)a The gradient of a penalty function. Parameters ---------- params : array_like A vector of parameters Returns ------- The gradient of the penalty with respect to each element in `params`. rrs r derivz Penalty.deriv;rrcrtj|jdkrt|dkr|jS)zwork around for Null model This will not be needed anymore when we can use `self._null_drop_keys` as in DiscreteModels. TODO: check other models )npsizer lenrs r _null_weightszPenalty._null_weightsKs9 74< 1 $ $6{{a|rNr)__name__ __module__ __qualname____doc__rrrrrr rrs_  """ """     rrc4eZdZdZfdZdZdZdZxZS) NonePenaltyz+ A penalty that does not penalize. c t|rddl}|ddSdS)Nrz keyword arguments are be ignored)superrwarningswarn)r kwdsr) __class__s r rzNonePenalty.__init___sM   > OOO MM< = = = = = > >rc^|jdkr!tj|jddSdS)Nrr)ndimrzerosshapers r rzNonePenalty.funces/ ;!  8FL,-- -1rc4tj|jSNrr0r1rs r rzNonePenalty.derivksx %%%rc@tj|jdS)Nrr4rs r deriv2zNonePenalty.deriv2nsx Q(((r r r!r"r#rrrr6 __classcell__r,s@r r&r&Zso>>>>>  &&&)))))))rr&c6eZdZdZdfd ZdZdZdZxZS)L2z! The L2 (ridge) penalty. rcJt|dSr3)r(r)r r r,s r rz L2.__init__xs! !!!!!rcPtj|j|jz|dzzSNr.)rsumr r rs r rzL2.func{s$vdlTZ/&!);<<rrs r rzL2.deriv~s4<$*,v55rcjd|jz|jztjt |zSr>)r r ronesrrs r r6z L2.deriv2s+4<$*,rws6{{/C/CCCrrr7r9s@r r;r;ss{""""""===666DDDDDDDrr;c,eZdZdZddZdZdZdZdS) L2Univariatez; The L2 (ridge) penalty applied to each parameter. Nc*| d|_dS||_dSrr r s r rzL2Univariate.__init__s ?DLLL"DLLLrc|j|dzzSr>rFrs r rzL2Univariate.funcs|fai''rcd|jz|zSr>rFrs r rzL2Univariate.derivs4<&((rcZd|jztjt|zSr>)r rrBrrs r r6zL2Univariate.deriv2s$4<"'#f++"6"666rr3)r r!r"r#rrrr6r$rr rDrDs_#### ((()))77777rrDc6eZdZdZdfd ZdZdZdZxZS) PseudoHuberz# The pseudo-Huber penalty. rcXt|||_dSr3)r(rdlt)r rMr r,s r rzPseudoHuber.__init__s& !!!rctjd||jz dzz}|dz}||jdzz}tj|j|jz|zdS)Nrr.r)rsqrtrMr?r r r rvs r rzPseudoHuber.funcs[ GA$(*Q.. / / Q TXq[vdlTZ/!3Q777rcptjd||jz dzz}||jz|jz|z S)Nrr.)rrOrMr r rPs r rzPseudoHuber.derivs; GA$(*Q.. / / $tz1A55rcltjd||jz dzzd}|j|jz|zS)Nrr.g)rpowerrMr r rPs r r6zPseudoHuber.deriv2s8 HQ&48+a// 6 6|dj(1,,rrr7r9s@r rKrKst888 666-------rrKc6eZdZdZdfd ZdZdZdZxZS) SCADu The SCAD penalty of Fan and Li. The SCAD penalty is linear around zero as a L1 penalty up to threshold tau. The SCAD penalty is constant for values larger than c*tau. The middle segment is quadratic and connect the two segments with a continuous derivative. The penalty is symmetric around zero. Parameterization follows Boo, Johnson, Li and Tan 2011. Fan and Li use lambda instead of tau, and a instead of c. Fan and Li recommend setting c=3.7. f(x) = { tau |x| if 0 <= |x| < tau { -(|x|^2 - 2 c tau |x| + tau^2) / (2 (c - 1)) if tau <= |x| < c tau { (c + 1) tau^2 / 2 if c tau <= |x| Parameters ---------- tau : float slope and threshold for linear segment c : float factor for second threshold which is c * tau weights : None or array weights for penalty of each parameter. If an entry is zero, then the corresponding parameter will not be penalized. References ---------- Buu, Anne, Norman J. Johnson, Runze Li, and Xianming Tan. "New variable selection methods for zero‐inflated count data with applications to the substance abuse field." Statistics in medicine 30, no. 18 (2011): 2326-2340. Fan, Jianqing, and Runze Li. "Variable selection via nonconcave penalized likelihood and its oracle properties." Journal of the American statistical Association 96, no. 456 (2001): 1348-1360. 皙 @rcft|||_||_dSr3)r(rtauc)r rYrZr r,s r rz SCAD.__init__s- !!!rc|j}tjtj|}tj|j|j}|tj||k}||j |zk}|||z||<||z}||}|dzd|j z|z|zz |dzz} | d|j dz zz ||<|j dz|dzzdz ||<|j |z dS)Nr.rg@r) rYr atleast_1dabsemptyr1dtypefillnanrZr r?) r rrYp_absresmask1mask3mask2p_abs2tmps r rz SCAD.funcsh bfVnn--hu{EK00  #%5<'E %uqy1tv:+f44sAv=TQ$&1*-.E fqjCF*R/E  s"''***rc|j}tj|}tj|}tj|}tj|j}|tj||k}||j |zk}||z} |||z||<|| || |j |zz z} | |j dz z || <d||<|j |zS)Nrr) rYrr\r]signr^r1r`rarZr ) r rrYprbp_signrcrdrerfrhs r rz SCAD.derivsh M& ! !q hu{##  #%%E]S(E UmuU|dfsl:;TTVaZ(E E |c!!rc|j}tj|}tj|}tj|j}||k}||j|zk}||z}d|jdz z ||<|j|zS)zSecond derivative of function This returns scalar or vector in same shape as params, not a square Hessian. If the return is 1 dimensional, then it is the diagonal of the Hessian. r)rYrr\r]r0r1rZr ) r rrYrkrbrcrdrerfs r r6z SCAD.deriv2sh M& ! !q hu{## #%%46A:&E |c!!r)rWrr7r9s@r rVrVsu&&P +++$"""("""""""rrVcBeZdZdZd fd ZfdZfdZfdZxZS) SCADSmoothedad The SCAD penalty of Fan and Li, quadratically smoothed around zero. This follows Fan and Li 2001 equation (3.7). Parameterization follows Boo, Johnson, Li and Tan 2011 see docstring of SCAD Parameters ---------- tau : float slope and threshold for linear segment c : float factor for second threshold c0 : float threshold for quadratically smoothed segment restriction : None or array linear constraints for Notes ----- TODO: Use delegation instead of subclassing, so smoothing can be added to all penalty classes. rWNrct|||||_||_||n|dz|_|j|krt d|j}|j}d|_t|}t|}||_|d|z|zz |_ d|z|z |_ ||_ dS)N)rZr g?zc0 cannot be larger than taurg?) r(rrYrZc0 ValueErrorr rraq1aq2 restriction) r rYrZrrr rvderiv_c0value_c0r,s r rzSCADSmoothed.__init__6s 7333""C#I 7S==;<< <W, 77==$$77<<## cHnr11>B&&rc||}|j2tj|dkr|j|}|j}d|_t |d}||_||jz}tj tj |}||j k}||}|j |dzz||<||z dS)Nrr)N.r.r)rrvrrdotr r(rrtr\r]rrrur?) r rr self_weightsvaluerbmask p_abs_maskedr,s r rzSCADSmoothed.funcKs$$V,,   'BGFOOa,?,?%))&11F|   VI.//#   bfVnn--twT{ hq0d %$$Q'''rc||}|j2tj|dkr|j|}|j}d|_t |}||_tj|}tj ||j k}d|j z||z||<|j5tj|dkr|||jzS||zSNrrr.) rrvrrrzr r(rr\r]rrrur rr r{r|rkr}r,s r rzSCADSmoothed.derivbs$$V,,   'BGFOOa,?,?%))&11F|   f%%#  M& ! !vayy47"$(lQtW,d   'BGFOOa,?,?UYYt'7888 8U? "rc$||}|j2tj|dkr|j|}|j}d|_t |}||_tj|}tj ||j k}d|j z||<|jBtj|dkr*|jj ||zz|jS||zSr) rrvrrrzr r(r6r\r]rrruTrs r r6zSCADSmoothed.deriv2ws$$V,,   'BGFOOa,?,?%))&11F|  v&&#  M& ! !vayy47"$(ld   'BGFOOa,?,?$&'E/:c$*++ ,U? "r)rWNrNr7r9s@r rprps2''''''*(((((.#####*#########rrpc0eZdZdZddZdZdZeZdZdS)ConstraintsPenaltya Penalty applied to linear transformation of parameters Parameters ---------- penalty: instance of penalty function currently this requires an instance of a univariate, vectorized penalty class weights : None or ndarray weights for adding penalties of transformed params restriction : None or ndarray If it is not None, then restriction defines a linear transformation of the parameters. The penalty function is applied to each transformed parameter independently. Notes ----- `restrictions` allows us to impose penalization on contrasts or stochastic constraints of the original parameters. Examples for these contrast are difference penalities or all pairs penalties. Ncp||_|d|_n||_|tj|}||_dSr)penaltyr rasarrayrv)r rr rvs r rzConstraintsPenalty.__init__sA ?DLL"DL  "*[11K&rc|j|j|}|j|}|j|jzjdS)aevaluate penalty function at params Parameter --------- params : ndarray array of parameters at which derivative is evaluated Returns ------- deriv2 : ndarray value(s) of penalty function Nr)rvrzrrr rr?r rr|s r rzConstraintsPenalty.funcsX   '%))&11F !!&)) uw&)--a000rc|j|j|}|j|}|j'|j|j|jzS|j|jzS)a#first derivative of penalty function w.r.t. params Parameter --------- params : ndarray array of parameters at which derivative is evaluated Returns ------- deriv2 : ndarray array of first partial derivatives )rvrzrrr rrs r rzConstraintsPenalty.derivsn   '%))&11F ""6**   '<%'++d.>"?"?? ?L57* +rc&|j|j|}|j|}|j2|jj|z|jz}||j}nt j|j|z}|S)asecond derivative of penalty function w.r.t. params Parameter --------- params : ndarray array of parameters at which derivative is evaluated Returns ------- deriv2 : ndarray, 2-D second derivative matrix )rvrzrr6rr rdiag)r rr|rQs r r6zConstraintsPenalty.deriv2s   '%))&11F ##F++   '!#e+dl:AEE$*++EEGDL5011E r)NN) r r!r"r#rrrgradr6r$rr rrsf. ' ' ' '111,,,,. Drrc$eZdZdZdfd ZxZS)L2ConstraintsPenaltyzAconvenience class of ConstraintsPenalty with L2 penalization Nc|tdt}t|||dS)Nz"sigma_prior is not implemented yet)r rv)rrDr(r)r r rv sigma_priorrr,s r rzL2ConstraintsPenalty.__init__sX  "%&JKK K.. '>I  K K K K Kr)NNN)r r!r"r#rr8r9s@r rrsQKKKKKKKKKKrrc eZdZdZdZdZdS)CovariancePenaltyc||_dSr3)weight)r rs r rzCovariancePenalty.__init__s  rct)z Parameters ---------- mat : square matrix The matrix to be penalized. mat_inv : square matrix The inverse of `mat`. Returns ------- A scalar penalty value rr matmat_invs r rzCovariancePenalty.funcs "!rct)a[ Parameters ---------- mat : square matrix The matrix to be penalized. mat_inv : square matrix The inverse of `mat`. Returns ------- A vector containing the gradient of the penalty with respect to each element in the lower triangle of `mat`. rrs r rzCovariancePenalty.deriv&s "!rN)r r!r"rrrr$rr rrsA " " """"""rrceZdZdZdZdZdS)PSDz A penalty that converges to +infinity as the argument matrix approaches the boundary of the domain of symmetric, positive definite matrices. c tj|}n&#tjj$rtjcYSwxYwd|jztjtjtj|zS)N) rlinalgcholesky LinAlgErrorinfrr?logr)r rrcys r rzPSD.func?sw ##C((BBy$   6MMM DK"& )<)<"="===s" AAc|}d|ztjtj|z }tj|jd\}}|j |||fzS)Nr.r)copyrr tril_indicesr1r)r rrrijs r rz PSD.derivFs_ \\^^ rTBGBGBKK(( (ocil++! |b1g%%rN)r r!r"r#rrr$rr rr8s< >>>&&&&&rr)r#numpyrrr&r;rDrKrVrprrrrr$rr rs(????????D)))))')))2DDDDDDDD$777777777*-----'---0g"g"g"g"g"7g"g"g"Tq#q#q#q#q#4q#q#q#hooooooood K K K K K- K K K$"$"$"$"$"$"$"$"N&&&&& &&&&&r