M/Ph2 dZddlZddlmZddlmZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZedkrdZerdZejdZejddZeddddfZejeeeedddfzzZgdZee eeeeejeez dddfezZeeeeeejeede eeejeez ddlm Z m!Z!dZ"dZ#dZ$dZ%dZ&dZ'eredee!j(e"dddd d!ed"edddeded#edddedee!j(e"d$d%dd&d!ed#ed$dded'ee!j(e"d$d%dd(d!ed#ed$dded'ee!j(e"d$d%dd)d!ed#ed$dded'ee!j(e#dd%dd*d!ee!j(e$dd%dd+d!d,\ZZ)Z*Z+ed#eee)e*ed'ee!j(e#e)d%dee*e+fd!ee!j(e$e*d%dee)e+fd!ed-ee!j(e%dddd.d!eeddde d/\ZZ)Z*ed#eee)e*e ee!j(e&e)d%dee*fd!ee!j(e'e*d%dee)fd!d0\ZZ)Z*ed#eee)e*e ee!j(e&e)d%dee*fd!ee!j(e'e*d%dee)fd!ed1ee dej,ddgd2e"dddd2d3ee dej,ddgde"ddddd3eedej,ddgde"ddddee dej,dd4gde"d5dd5deedej,ddgd2e"dddd2ee dej,ddgde"ddd5ej-d6zddd7l.m/Z/gd8Zej0d9d:d;Z1eD]m\Z)Z*e!j(e&e)d%de1e*fd!Z2e!j(e'e*d%de1e)fd!Z3ee1e)e*e Z4e/e2e4dde/e3e4ddnd@D]Z+eD]\Z)Z*e!j(e#e)d%de1e*e+fd!Z2e!j(e$e*d%de1e)e+fd!Z3ee1e)e*ee+Z4e/e2e4ddAd=>e/e3e4ddAd?>e/ee1ej,e)e*dzge+e"e1e)e*e+de/e e1ej,e)e*dzge+e"e1e)e*ej-e+d:z e+z ze+ddSdS)Cagradient/Jacobian of normal and t loglikelihood use chain rule normal derivative wrt mu, sigma and beta new version: loc-scale distributions, derivative wrt loc, scale also includes "standardized" t distribution (for use in GARCH) TODO: * use sympy for derivative of loglike wrt shape parameters it works for df of t distribution dlog(gamma(a))da = polygamma(0,a) check polygamma is available in scipy.special * get loc-scale example to work with mean = X*b * write some full unit test examples A: josef-pktd N)special)gammalnc|j\}}dtjdtjztj|z||z dz|z zz}|S)aynormal loglikelihood given observations and mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows Returns ------- lls : ndarray contribution to loglikelihood for each observation g)Tnplogpi)yparamsmusigma2llss d/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/sandbox/regression/tools.pynorm_llsrsK JB qw"&..0AbD19V3CC DC Jc|j\}}||z |z }||z dz|z dz tj|z }tj||fS)aBJacobian of normal loglikelihood wrt mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows Returns ------- grad : array (nobs, 2) derivative of loglikelihood for each observation wrt mean in first column, and wrt variance in second column Notes ----- this is actually the derivative wrt sigma not sigma**2, but evaluated with parameter sigma2 = sigma**2 r)rrsqrt column_stack)r r r rdllsdmu dllsdsigma2s r norm_lls_gradr/sU,JBtVmGbD19V#a'8K ?G[1 2 22rc|S)z-gradient/Jacobian for d (x*beta)/ d beta )xbetas r mean_gradrKs  Hrcj|dd}|dtjt|dfz}t||}tj||}tj||f}t ||}tj|ddddf|z|ddddff} | S)aJacobian of normal loglikelihood wrt mean mu and variance sigma2 Parameters ---------- y : ndarray, 1d normally distributed random variable with mean x*beta, and variance sigma2 x : ndarray, 2d explanatory variables, observation in rows, variables in columns params : array_like, (nvars + 1) array of coefficients and variance (beta, sigma2) Returns ------- grad : array (nobs, 2) derivative of loglikelihood for each observation wrt mean in first column, and wrt scale (sigma) in second column assume params = (beta, sigma2) Notes ----- TODO: for heteroscedasticity need sigma to be a 1d array Nr)roneslenrdotrr) r rr rrdmudbetar params2dllsdmsgrads rnormgradr(Ps0 #2#;D BZQ ++ +FD!!H 4Bor&k**GAg&&G ?GAAAbqbDM(2GAAAbqbDMB C CD KrcX|j\}}|dz}t|dzdz t|dz z dtj|dz tjzzz }||dzdz tjd||z dz|dz z |z zzdtj|zzz}|S)at loglikelihood given observations and mean mu and variance sigma2 = 1 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows df : int degrees of freedom of the t distribution Returns ------- lls : ndarray contribution to loglikelihood for each observation Notes ----- parametrized for garch ?r@?r)rrrr r r r dfr rrs rtstd_llsr/ts,JB CB 2a4)  wr"u~~ -BFBqD"%<4H4H0H HCBqD"9rvbAbD19bd#3F#::;; ;cBF6NN>R RRC Jrc| S)z?derivative of log pdf of standard normal with respect to y r)r s r norm_dlldyr1s  2Irc:tj|dz}tjtj|dzdz tj|dz z tj|dz tjzz }|d|dz|dz z z|dzdz zz}|S)zIpdf for standardized (not standard) t distribution, variance is one r*rr+r)rarrayexprrrr )rr.rPxs rtstd_pdfr7s CA 1b))'/!B$*?*?? @ @!A#ruAUAU UB1adQqS\>ac2X &&B Ircnt||||j\}}|dz}t|dzdz t|dz z dtj|tjzzz }||dzdz tjd||z dz|z |z zzdtj|zzz}|S)at loglikelihood given observations and mean mu and variance sigma2 = 1 Parameters ---------- y : ndarray, 1d normally distributed random variable params : ndarray, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows df : int degrees of freedom of the t distribution Returns ------- lls : ndarray contribution to loglikelihood for each observation Notes ----- parametrized for garch normalized/rescaled so that sigma2 is the variance >>> df = 10; sigma = 1. >>> stats.t.stats(df, loc=0., scale=sigma.*np.sqrt((df-2.)/df)) (array(0.0), array(1.0)) >>> sigma = np.sqrt(2.) >>> stats.t.stats(df, loc=0., scale=sigma*np.sqrt((df-2.)/df)) (array(0.0), array(2.0)) r*rr+r,r)printrrrr r r-s rts_llsr:s: !VRJB CB 2a4)  wr"u~~ -BFB:4F4F0F FCBrE2:rQrTAIrN6$99:: :S26&>>=Q QQC Jrc<|dz}|dz |z d|dz|z zz |zS)aderivative of log pdf of standard t with respect to y Parameters ---------- y : array_like data points of random variable at which loglike is evaluated df : array_like degrees of freedom,shape parameters of log-likelihood function of t distribution Returns ------- dlldy : ndarray derivative of loglikelihood wrt random variable y evaluated at the points given in y Notes ----- with mean 0 and scale 1, but variance is df/(df-2) r*rrrr r.s rts_dlldyr=s5, BBT7B<1q!tRy= )A --rc>|dz |dz z d|dz|dz z zz |zS)aderivative of log pdf of standardized t with respect to y Parameters ---------- y : array_like data points of random variable at which loglike is evaluated df : array_like degrees of freedom,shape parameters of log-likelihood function of t distribution Returns ------- dlldy : ndarray derivative of loglikelihood wrt random variable y evaluated at the points given in y Notes ----- parametrized for garch, standardized to variance=1 rr+rrr<s r tstd_dlldyr?s3.T7BrE?a!Q$2,. /! 33rcj||z |z }||g|R |z }d|z ||g|R||z z|dzz z }||fS)aderivative of log-likelihood with respect to location and scale Parameters ---------- y : array_like data points of random variable at which loglike is evaluated loc : float location parameter of distribution scale : float scale parameter of distribution dlldy : function derivative of loglikelihood fuction wrt. random variable x args : array_like shape parameters of log-likelihood function Returns ------- dlldloc : ndarray derivative of loglikelihood wrt location evaluated at the points given in y dlldscale : ndarray derivative of loglikelihood wrt scale evaluated at the points given in y grr)r locscaledlldyargsystdlldloc dlldscales r locscale_gradrHsm4 S5%-CuS 4    5(GE EE#----37q@@I I r__main__g?r r)rrr)statsmisccltjtj||||SN)rArBrr rLtpdf)r rArBr.s rlltrS2(vegkk!RSk>>???rcltjtj||||SrOrP)rAr rBr.s rlltlocrV4rTrcltjtj||||SrOrP)rBr rAr.s rlltscalerX6rTrcjtjtj|||SrOrr rLnormrR)r rArBs rllnormr\9&vejnnQCun==>>>rcjtjtj|||SrOrZ)rAr rBs r llnormlocr_;r]rcjtjtj|||SrOrZ)rBr rAs r llnormscalera=r]rz gradient of tgư>)rrrJ)dxnrDorderzt ts?g|=)rrrg)rrrg)rrrg)rfrrg)rfrrg)rfrrrgz gradient of normrr)rfrr)rfrrz loglike of tdzdifferently standardizedg?r*g?)assert_almost_equal)rh)r*r*)gr+)r*r+gr+ z deriv loc)err_msgz deriv scale)rKrJriloglike)5__doc__numpyrscipyr scipy.specialrrrrr(r/r1r7r:r=r?rH__name__verbosesigr!rrandomrandnrvsrr#r r r9 dllfdbetarLrMrSrVrXr\r_ra derivativerArBr.r3r numpy.testingrjlinspaceytdlldlodlldscgrrrrrsn *!!!!!!(3338     H> $$$N...64444> zG!rwqzzioob## !""I BF1TNNSQQQqS\ ) hhq!V$$%%%vrva&$/1  i mmAvrva:>>??? aq$   !!!!!!!!@@@@@@@@@?????????&O     odoc1KKKLLL dMM!Q:r::;;; dMM!Q8R88999 odoc35AHANNNPPP dMM#q!Xr::;;; odoc35AHANNNPPP dMM#q!Xr::;;; odoc35AHANNNPPP dMM#q!Xr::;;; odofaEQZqQQQSSS odoheqzQRSSSTTT&#eB dMM!Cx<<=== odofceq%|STUUUWWW odoh%1Ac":UVWWWXXX "### odofaDAEKKKLLL mmAq!Z00111 #e dMM!Cz::;;; odoi!1U)STUUUWWW odok5UaqgUVWWWXXX #e dMM!Cz::;;; odoi!1U)STUUUWWW odok5UaqgUVWWWXXX  hhq("(Aa5//3//Qq3A[\\\ hhq("(Aa5//2..Aa" ?YZZZ ffQ!A,,cc!Aamm<<< hhq("(Ai=11266BqBHHH ffQ!A--ss1Qq~~>>> hhq("(Aa5//2..Aa7275>>8I"0M0MNNN211111 1 1 1F SB  BEE E CEQbZWXYYY e"SYZ[[[ ]2sE: 6 6FBqE1kBBBBFBqE1mDDDDD 3 3 3 3IC$T_VSUar%PRm[\]]]F$T_Xu!2cRT+]^___Fr3x<