M/Phu[ dZddlZddlmZmZmZiZdqdZdqdZ dqdZ drdZ drdZ dsd Z dqd Zdtd Z d ZdZdZdqdZedk rededdZddgZddgZeejeeejeeZeejejeejddddddfedddzeejddejZeeddgejedd d!Zeejed d"gejedd Z eejd#d d"gej!edd"d$%Z"ej#e"$e"%gZ ejd&d dgZ&ee&ed'eejee&z ed(ed)d*ej'j(_)ej#gd+Zej*!dd,d-%Z+ed.dd,dd/eej*,e+d0ed1zDZ-ej#e+$e+%gZ.ej/d2gd3d-4Z0ed5e0ed6ed7d$Z1ej2Zej#ddgZd8Z3e!d9dd/e1%Z4d:ed1zDZ5eje3Z6ej#e4$e4%gZ.ejd;gd<Z7ed=e7ej/d>gd<Z8ed?e8ee,e4eejd@d d"geej/dAd d"gee,e4eejdBd d"gej#gd+ZdCed1zDZ5ej/dDgd<Z9edEe9e ee4gd<ddFZ:edGe:ej2ejdd/dHd9Z;ej<e4e;I\Z=Z;edJeej2e=e;gdKZ>eej2e=e;gdKZ?eej2e=e;gdKdLMZ@ee3ej<e4dNI\ZAZB eej2eAeBgdKZCeej2eAeBgdKZDeej2eAeBgdKdLMZEej2,e4ZFejGdOPedQe1edRe3edSeFeedTeCedUeDedVeEeedWe>edXe?edYe@eedZe9ed[e7ed\e8 ed]ed^ejHdZIdZIeeIejJ!eId1dd_`ZKeeKLeejJ,eKd eKLd/z d/aejMeKZNd eKLd/z d/gZOeNejJfZPeej/eeOePbedceddejQ!dde%ZRejMeRZSeSejTfZUd eRLd9z d/gZVeej/eeVeUbeejT,eRddfdaedgejQ,eReejQeRZWedheWeSejQfZXeYdijZZeeeWeXeZ\Z[Z\eej]^e[ddkZ_ee_eWej<eRdNI\Z`Za eejQe`eaeVZbeejQe`eaeVZcedlebedmece ejQeReVddFZdednedee ejQeReVejddoddFej<eRejQdejddodI\Z`ZaedpeeejQe`eaeVdSdS)ua(estimate distribution parameters by various methods method of moments or matching quantiles, and Maximum Likelihood estimation based on binned data and Maximum Product-of-Spacings Warning: I'm still finding cut-and-paste and refactoring errors, e.g. hardcoded variables from outer scope in functions some results do not seem to make sense for Pareto case, looks better now after correcting some name errors initially loosely based on a paper and blog for quantile matching by John D. Cook formula for gamma quantile (ppf) matching by him (from paper) http://www.codeproject.com/KB/recipes/ParameterPercentile.aspx http://www.johndcook.com/blog/2010/01/31/parameters-from-percentiles/ this is what I actually used (in parts): http://www.bepress.com/mdandersonbiostat/paper55/ quantile based estimator ^^^^^^^^^^^^^^^^^^^^^^^^ only special cases for number or parameters so far Is there a literature for GMM estimation of distribution parameters? check found one: Wu/Perloff 2007 binned estimator ^^^^^^^^^^^^^^^^ * I added this also * use it for chisquare tests with estimation distribution parameters * move this to distribution_extras (next to gof tests powerdiscrepancy and continuous) or add to distribution_patch example: t-distribution * works with quantiles if they contain tail quantiles * results with momentcondquant do not look as good as mle estimate TODOs * rearange and make sure I do not use module globals (as I did initially) DONE make two version exactly identified method of moments with fsolve and GMM (?) version with fmin and maybe the special cases of JD Cook update: maybe exact (MM) version is not so interesting compared to GMM * add semifrozen version of moment and quantile based estimators, e.g. for beta (both loc and scale fixed), or gamma (loc fixed) * add beta example to the semifrozen MLE, fitfr, code -> added method of moment estimator to _fitstart for beta * start a list of how well different estimators, especially current mle work for the different distributions * need general GMM code (with optimal weights ?), looks like a good example for it * get example for binned data estimation, mailing list a while ago * any idea when these are better than mle ? * check language: I use quantile to mean the value of the random variable, not quantile between 0 and 1. * for GMM: move moment conditions to separate function, so that they can be used for further analysis, e.g. covariance matrix of parameter estimates * question: Are GMM properties different for matching quantiles with cdf or ppf? Estimate should be the same, but derivatives of moment conditions differ. * add maximum spacings estimator, Wikipedia, Per Brodtkorb -> basic version Done * add parameter estimation based on empirical characteristic function (Carrasco/Florens), especially for stable distribution * provide a model class based on estimating all distributions, and collect all distribution specific information References ---------- Ximing Wu, Jeffrey M. Perloff, GMM estimation of a maximum entropy distribution with interval data, Journal of Econometrics, Volume 138, Issue 2, 'Information and Entropy Econometrics' - A Volume in Honor of Arnold Zellner, June 2007, Pages 532-546, ISSN 0304-4076, DOI: 10.1016/j.jeconom.2006.05.008. http://www.sciencedirect.com/science/article/B6VC0-4K606TK-4/2/78bc07c6245546374490f777a6bdbbcc http://escholarship.org/uc/item/7jf5w1ht (working paper) Johnson, Kotz, Balakrishnan: Volume 2 Author : josef-pktd License : BSD created : 2010-04-20 changes: added Maximum Product-of-Spacings 2010-05-12 N)statsoptimizespecialcfd}|S)aestimate distribution parameters based method of moments (mean, variance) for distributions with 1 shape parameter and fixed loc=0. Returns ------- cond : function Notes ----- first test version, quantile argument not used cj|\}}|d|}tj|z S)Nrnparray)paramsalphascalemom2sdistfnmom2s l/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/sandbox/distributions/estimators.pycondzgammamomentcond..condns4 u UBu--x~~e##)rr rquantilers` ` rgammamomentcondras)$$$$$$ Krch|\}}||d|}tj||z S)aestimate distribution parameters based method of moments (mean, variance) for distributions with 1 shape parameter and fixed loc=0. Returns ------- difference : ndarray difference between theoretical and empirical moments Notes ----- first test version, quantile argument not used The only difference to previous function is return type. rr )rr rrr rrs rgammamomentcond2rus5 LE5 LL5 ) )E 8D>>% rc|\}}}tj|||||z }|>|\}} || ||||z } tj|| ddgS|S)a+moment conditions for estimating distribution parameters using method of moments, uses mean, variance and one quantile for distributions with 1 shape parameter. Returns ------- difference : ndarray difference between theoretical and empirical moments and quantiles N)r r rcdf concatenate rr rrshapelocrmom2diffpqxqcdfdiffs rmomentcondunboundr%sE3x UC6677$>HB**RU33b8~x!5666 Orc|\}}tj|||||z }|"|\}} || ||||z } | S|S)amoment conditions for estimating loc and scale of a distribution with method of moments using either 2 quantiles or 2 moments (not both). Returns ------- difference : ndarray difference between theoretical and empirical moments or quantiles )r r rrrs rmomentcondunboundlsr'sfJCx UC77884?HB**RU33b8 Orct|dkr|\}}nt|dkr|\}}}n |\}}|j|g|R|z } | S)admoment conditions for estimating distribution parameters by matching quantiles, defines as many moment conditions as quantiles. Returns ------- difference : ndarray difference between theoretical and empirical quantiles Notes ----- This can be used for method of moments or for generalized method of moments. )lenr) rr rrrr rr"r#r$s rmomentcondquantr,sr  6{{a UU V  "sEE FBfj%f%%%*G Nrc tjgd|6tdr}ndgjzddgz}fddzDdt jfd|}|S) N{Gz?g?皙?g?g333333??gffffff?Gz? _fitstartrr?c:g|]}tj|Sr)rscoreatpercentile).0pxs r z#fitquantilesgmm..s& = = =Q5 "1a ( ( = = =rdc Ztjt|fddzSNrr))r sumr,)r rrpquantxqss rz!fitquantilesgmm..s4vcl$GGGJ*L*Lr)r r hasattrr3numargsrfmin)rr9startr@frozenparestrrAs`` ` @@rfitquantilesgmmrIs ~@@@AA } 6; ' ' 1$$Q''EEC&"R0E = = = =&* = = =C E ]LLLLLLLMRTTF Mrc|ttj}tj|dzfd}t j||S)aMestimate parameters of distribution function for binned data using MLE Parameters ---------- distfn : distribution instance needs to have cdf method, as in scipy.stats freq : ndarray, 1d frequency count, e.g. obtained by histogram binedges : ndarray, 1d binedges including lower and upper bound start : tuple or array_like ? starting values, needs to have correct length Returns ------- paramest : ndarray estimated parameters Notes ----- todo: add fixed parameter option added factorial Nrctjjg|R}tjtj|zt jdzz z S)[negative loglikelihood function of binned data corresponds to multinomial r)r diffrr?logrgammaln)r probbinedgesrfreq lnnobsfacts rnloglikezfitbinned..nloglike s` wzvz(4V44455bfT"&,,%6Q8O8O%OPPPQQr)NotImplementedErrorr r?rrOrrE)rrRrQrFfixednobsrTrSs``` @r fitbinnedrXsx4 !! 6$<.gmmobjective?sH wzvz(4V44455t#vggow///r)rUr r?floatonesr+rrE) rrRrQrFrVweightsoptimalrWr_r]r^s ` ` @@r fitbinnedgmmrcs> !! 6$<c|gRSNr)r argsfuns rrBzhess_ndt..Xsss6)D)))r) numdifftoolsHessian)rjparsrioptionsndtfhs` ` rhess_ndtrrTsk  I$8$8! )))))A A!!!!A 1T77A:rctjd|j|g|Rdf}tj|}tj| S)acalculate negative log of Product-of-Spacings Parameters ---------- params : array_like, tuple ? parameters of the distribution funciton xsorted : array_like data that is already sorted dist : instance of a distribution class only cdf method is used Returns ------- mps : float negative log of Product-of-Spacings Notes ----- MPS definiton from JKB page 233 rr4)r r_rrMrNmean)r xsorteddistxcdfDs rlogmpsrz\sS, 5XTXg////3 4D  A F1IINN   rcRt|dr||}ntj|jr4tjdg|jz|dz df}n3tjdg|jz|dz df}|S)aget starting values for estimation of distribution parameters Parameters ---------- dist : distribution instance the distribution instance needs to have either a method fitstart or an attribute numargs data : ndarray data for which preliminary estimator or starting value for parameter estimation is desired Returns ------- x0 : ndarray preliminary estimate or starting value for the parameters of the distribution given the data, including loc and scale fitstartr4r) rCr|r isfiniteartrDminru)rwdatax0s rgetstartparamsrvs&tZ  ? ]]4  ;tv   ?tDL(488::a<"<=BBtDL(499;;q=2=>B Irctj|}|t||}||f}t|t jt ||S)aEstimate distribution parameters with Maximum Product-of-Spacings Parameters ---------- params : array_like, tuple ? parameters of the distribution funciton xsorted : array_like data that is already sorted dist : instance of a distribution class only cdf method is used Returns ------- x : ndarray estimates for the parameters of the distribution given the data, including loc and scale Nri)r sortrprintrrErz)rwrrrvris rfit_mpsrsP(gdmmG z D' * * T?D "III =$ / / //r__main__z Example: gamma Distributionz---------------------------r)g?r0r1r/ ctjtjt |t dddzS)Nr)r rMrgammappfr"r#)r s rrBrBs0 E0J0J2ddPRd80S(T(Trg@g@rr4)rr@c8tt|tSrhrrrr s rrBrBs(8(N(Nri)sizec8tt|tSrhrrs rrBrBs.>vvt.T.Trzscale = z Example: beta Distributionz--------------------------cdS)N)rrrr)selfrs rrBrBsrr.iz true paramsrcBg|]}tjt|Sr)rr6rvsbr7r8s rr:r:s% = = =E #D! , , = = =rr;c tjttj|t t tfddzSr=)r r?r,rbetarr"xqsbrs rrBrBsQPUPZ\bdikmnrjs{AAAAAACDAD:E:Er)rrrr4)maxiterbetaparest_gmmquantilez Example: t Distributionz-----------------------)rrrrcBg|]}tjt|Srrr6trvsrs rr:r:% < < <5 "4 + + < < OPVW]_dfhilem>n>npq>q7r7rrtparest_gmm3quantilec<tt|tdSNrr>)r'rrrs rrBrBs(;FFEXY(Z(Z(Zrcftjtt|tddzS)Nrr>r))r r?r'rrrs rrBrB s*bf-@QV]^-_-_-_ab-b&c&crcVtt|tttfdSr)r'rrr"rArs rrBrB s"(;FFESUVYRZab(c(c(crcBg|]}tjt|Srrrs rr:r:rrc tjtt|tt t fddzSr=)r r?r,rrr"rArs rrBrBs3bf_VU[]bdfgjcksw=x=x=xz{={6|6|rtparest_gmmquantile)rFr@rGtparest_gmmquantile2)binszfitbinned t-distribution)rrrF)rb2) precisionz sample sizeztrue (df, loc, scale) zparest_mle ztparest_mlebinel ztparest_gmmbinelidentity ztparest_gmmbineloptimal ztparest_mlebinew ztparest_gmmbinewidentity ztparest_gmmbinewoptimal tparest_gmmquantileidentityztparest_gmm3quantilefsolve ztparest_gmm3quantile z! Example: Lognormal Distributionz-------------------------------)r rr)r rrz: Example: Lomax, Pareto, Generalized Pareto Distributionsz8--------------------------------------------------------iizgpdparest_ mlezgpdparest_ mpsgHz>)rfc&t|gtRSrh)rzargsgpdrs rrBrBsvf/w///rgpdparest_mlebinelgpdparest_gmmbinelidentitygpdparest_gmmquantile2r2zfitbinnedgmm equal weight binsrh)NN)NNN)NT)e__doc__numpyr scipyrrrcacherrr%r'r,rIrXrcrrrzrr__name__rr r#r"rrrMlinspacefsolvermcondrrvsgrvsr ruvar alphaestq distributionsbeta_genr3rrfitrrrErrWt paramsdgprrAmom2thrrrrbt histogramfttparest_mlebinewtparest_gmmbinewidentitytparest_gmmbinewoptimalft2bt2tparest_mlebineltparest_gmmbinelidentitytparest_gmmbineloptimal tparest_mleset_printoptionsexpshlognormr9rrrvrri genparetop2rvs p2rvssortedparetoargspx0pparsgpdrdictrnherqlinalgeighrpfp2bp2rrrrrrrsY WWr********** (    .,2B    "&*&*&*&*R1.1.1.1.h4:0000< z  E +,,, E+,,, E qB sB E%+//"e $ $%%% U # #B E'"'%+//"kbk$q&<&>"RT> * *D E-RA&&& E%*..   = =bf = = =D BHdiikk488::. / /E*X],E,E+:??DJJJ E "%;<<< E '((( E'((( D WF 3s)  BI ::aAD: ) )D < >>@ E /000 E/000 B B E"III "BC88A E!%%''NNN E%-  Aquuwwqyq  9 9:::bgajjG aeeggai B U] #D E-(-rt , , ,--- E HIII EHIII O    , ,E"'%..K %, 'E uyy{{1}a C E-(-s . . ./// E%,  5#3c  : :;;; E EO//66777geou--G E G$$$EO,Gd4   G HVWgw 7 7EB E")..  Q    //A E!!G**r|E+++HC"5?CcBB!-eosC!M!M E  2333 E &(BCCC,_ c$tEEE E "$:;;; E//%/5!,T$r!:!:4 I I IJJJr| U_Q   # #KBK$r$:$: ; ;===HC E *+++ E,,uS# 6 677777ir