M/PhddlmZddlZddlmZddlmZddlm Z ddl m Z ddl m Z dd lmZgd ZGd d Zejd ddddddfdZ ddZddZddZdZ ddZddZdS))lzipN)stats)ECDF)OLS)cache_readonly) add_constant)utils)qqplotqqplot_2samplesqqlineProbPlotceZdZdZejdddddfdZedZedZ ed Z ed Z ed Z dd Z ddefdZ ddZd S)ra Q-Q and P-P Probability Plots Can take arguments specifying the parameters for dist or fit them automatically. (See fit under kwargs.) Parameters ---------- data : array_like A 1d data array dist : callable Compare x against dist. A scipy.stats or statsmodels distribution. The default is scipy.stats.distributions.norm (a standard normal). Can be a SciPy frozen distribution. fit : bool If fit is false, loc, scale, and distargs are passed to the distribution. If fit is True then the parameters for dist are fit automatically using dist.fit. The quantiles are formed from the standardized data, after subtracting the fitted loc and dividing by the fitted scale. fit cannot be used if dist is a SciPy frozen distribution. distargs : tuple A tuple of arguments passed to dist to specify it fully so dist.ppf may be called. distargs must not contain loc or scale. These values must be passed using the loc or scale inputs. distargs cannot be used if dist is a SciPy frozen distribution. a : float Offset for the plotting position of an expected order statistic, for example. The plotting positions are given by (i - a)/(nobs - 2*a + 1) for i in range(0,nobs+1) loc : float Location parameter for dist. Cannot be used if dist is a SciPy frozen distribution. scale : float Scale parameter for dist. Cannot be used if dist is a SciPy frozen distribution. See Also -------- scipy.stats.probplot Notes ----- 1) Depends on matplotlib. 2) If `fit` is True then the parameters are fit using the distribution's `fit()` method. 3) The call signatures for the `qqplot`, `ppplot`, and `probplot` methods are similar, so examples 1 through 4 apply to all three methods. 4) The three plotting methods are summarized below: ppplot : Probability-Probability plot Compares the sample and theoretical probabilities (percentiles). qqplot : Quantile-Quantile plot Compares the sample and theoretical quantiles probplot : Probability plot Same as a Q-Q plot, however probabilities are shown in the scale of the theoretical distribution (x-axis) and the y-axis contains unscaled quantiles of the sample data. Examples -------- The first example shows a Q-Q plot for regression residuals >>> # example 1 >>> import statsmodels.api as sm >>> from matplotlib import pyplot as plt >>> data = sm.datasets.longley.load() >>> data.exog = sm.add_constant(data.exog) >>> model = sm.OLS(data.endog, data.exog) >>> mod_fit = model.fit() >>> res = mod_fit.resid # residuals >>> pplot = sm.ProbPlot(res) >>> fig = pplot.qqplot() >>> h = plt.title("Ex. 1 - qqplot - residuals of OLS fit") >>> plt.show() qqplot of the residuals against quantiles of t-distribution with 4 degrees of freedom: >>> # example 2 >>> import scipy.stats as stats >>> pplot = sm.ProbPlot(res, stats.t, distargs=(4,)) >>> fig = pplot.qqplot() >>> h = plt.title("Ex. 2 - qqplot - residuals against quantiles of t-dist") >>> plt.show() qqplot against same as above, but with mean 3 and std 10: >>> # example 3 >>> pplot = sm.ProbPlot(res, stats.t, distargs=(4,), loc=3, scale=10) >>> fig = pplot.qqplot() >>> h = plt.title("Ex. 3 - qqplot - resids vs quantiles of t-dist") >>> plt.show() Automatically determine parameters for t distribution including the loc and scale: >>> # example 4 >>> pplot = sm.ProbPlot(res, stats.t, fit=True) >>> fig = pplot.qqplot(line="45") >>> h = plt.title("Ex. 4 - qqplot - resids vs. quantiles of fitted t-dist") >>> plt.show() A second `ProbPlot` object can be used to compare two separate sample sets by using the `other` kwarg in the `qqplot` and `ppplot` methods. >>> # example 5 >>> import numpy as np >>> x = np.random.normal(loc=8.25, scale=2.75, size=37) >>> y = np.random.normal(loc=8.75, scale=3.25, size=37) >>> pp_x = sm.ProbPlot(x, fit=True) >>> pp_y = sm.ProbPlot(y, fit=True) >>> fig = pp_x.qqplot(line="45", other=pp_y) >>> h = plt.title("Ex. 5 - qqplot - compare two sample sets") >>> plt.show() In qqplot, sample size of `other` can be equal or larger than the first. In case of larger, size of `other` samples will be reduced to match the size of the first by interpolation >>> # example 6 >>> x = np.random.normal(loc=8.25, scale=2.75, size=37) >>> y = np.random.normal(loc=8.75, scale=3.25, size=57) >>> pp_x = sm.ProbPlot(x, fit=True) >>> pp_y = sm.ProbPlot(y, fit=True) >>> fig = pp_x.qqplot(line="45", other=pp_y) >>> title = "Ex. 6 - qqplot - compare different sample sizes" >>> h = plt.title(title) >>> plt.show() In ppplot, sample size of `other` and the first can be different. `other` will be used to estimate an empirical cumulative distribution function (ECDF). ECDF(x) will be plotted against p(x)=0.5/n, 1.5/n, ..., (n-0.5)/n where x are sorted samples from the first. >>> # example 7 >>> x = np.random.normal(loc=8.25, scale=2.75, size=37) >>> y = np.random.normal(loc=8.75, scale=3.25, size=57) >>> pp_x = sm.ProbPlot(x, fit=True) >>> pp_y = sm.ProbPlot(y, fit=True) >>> pp_y.ppplot(line="45", other=pp_x) >>> plt.title("Ex. 7A- ppplot - compare two sample sets, other=pp_x") >>> pp_x.ppplot(line="45", other=pp_y) >>> plt.title("Ex. 7B- ppplot - compare two sample sets, other=pp_y") >>> plt.show() The following plot displays some options, follow the link to see the code. .. plot:: plots/graphics_gofplots_qqplot.py Frr c||_||_|jd|_||_||_t |tjj |_ |j r#|s|dks |dks|dkrtdi|_ |j rk||_ |j }|j} | ;tt!t"j| d} nd} t)| } |j} t)| | dzkr| | |_n |jd||_t)| | dzkr| | dz|_n |jd||_g} t5| D]>\}}||jvr|j|}n |j|}| |?t8j| |j|jf|_dS|r|||_|jd |_|jd |_t)|jdkr+||jdd it?dd |_ dS|dd |_ dS|s |dks|dkr ||it?|| |_ nj#t@$r]d !d |D}d}|"|||}tGd"|wxYw||_||_t8j|||f|_dS||_ ||_||_t8j||f|_dS)Nrr rzIFrozen distributions cannot be combined with fit, loc, scale or distargs.,locscale)rrz, c,g|]}t|Srstr).0das ]/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/graphics/gofplots.py z%ProbPlot.__init__..s%A%A%A"c"gg%A%A%Az*dist({distargs}, loc={loc}, scale={scale}))distargsrrzInitializing the distribution failed. This can occur if distargs contains loc or scale. The distribution initialization command is: {cmd})cmd)$dataashapenobsr fit isinstancer distributions rv_frozen _is_frozen ValueError_cachedistshapestuplemaprstripsplitlenargsrkwdsgetr enumerateappendnpr_ fit_paramsdict Exceptionjoinformat TypeError)selfr"r-r&r r#rrdist_genr. shape_argsnumargsr4r;iargvaluer!s r__init__zProbPlot.__init__s JqM   $T5+>+HII ?   !88uzzX^^    ?6 0DIyH_F!"3sy&,,s2C2C#D#DEE  *ooG9D4yyGaK''=9==444yyGaK''!'A+. !Y]]7E:: J#J// ) )3$)## IcNEE IaLE!!%(((( eJ$*$DEDOOO  0"hhtnnDOr*DH,DJ4?##a'' D$/#2#"6O$1A:N:N:NOO  DQa000  0UaZZ  D(Ids%.H.H.HII    99%A%A%A%A%ABBBjj(5jII!"(C  DHDJ eHc5$89DOOODIDHDJ eCJ/DOOOs J::A'L!c6t|j|jS)zTheoretical percentiles) plotting_posr%r#rAs rtheoretical_percentilesz ProbPlot.theoretical_percentilessDItv...rc |j|jS#t$r|jjd}t|t $r,}d|jj}t ||d}~wwxYw)zTheoretical quantilesz( requires more parameters to compute ppfzfailed to compute the ppf of N)r-ppfrLr@namer=type)rAmsgexcs rtheoretical_quantileszProbPlot.theoretical_quantiless !9==!=>> > ! ! !Y^MMMCC..  ! ! !B$).BBC$s))C..  !s!1A>'A99A>cXtjtj|jS)z sorted data)r9sortarrayr"rKs r sorted_datazProbPlot.sorted_data s wrx **+++rcx|jr-|jdkr"|jdkr|j|jz |jz S|jS)zsample quantilesrr )r&rrrWrKs rsample_quantileszProbPlot.sample_quantilessB 8 $A $*//$tx/4:= =# #rct|jd|jr|j|jS|j|jdz |jdz }|j|S)zSample percentilescdfrr) _check_forr-r*r[rWr;)rA quantiless rsample_percentileszProbPlot.sample_percentilesss 49e$$$ ? 39==!122 2%(;;t @  y}}Y'''rNc |rt|t}|st|}|j}t|j|j} t || |jf||d|\} }|d}|d}n,t |j|j|jf||d|\} }|d}|d}||| || ddg| ddg| S) a Plot of the percentiles of x versus the percentiles of a distribution. Parameters ---------- xlabel : str or None, optional User-provided labels for the x-axis. If None (default), other values are used depending on the status of the kwarg `other`. ylabel : str or None, optional User-provided labels for the y-axis. If None (default), other values are used depending on the status of the kwarg `other`. line : {None, "45", "s", "r", q"}, optional Options for the reference line to which the data is compared: - "45": 45-degree line - "s": standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - "r": A regression line is fit - "q": A line is fit through the quartiles. - None: by default no reference line is added to the plot. other : ProbPlot, array_like, or None, optional If provided, ECDF(x) will be plotted against p(x) where x are sorted samples from `self`. ECDF is an empirical cumulative distribution function estimated from `other` and p(x) = 0.5/n, 1.5/n, ..., (n-0.5)/n where n is the number of samples in `self`. If an array-object is provided, it will be turned into a `ProbPlot` instance default parameters. If not provided (default), `self.dist(x)` is be plotted against p(x). ax : AxesSubplot, optional If given, this subplot is used to plot in instead of a new figure being created. **plotkwargs Additional arguments to be passed to the `plot` command. Returns ------- Figure If `ax` is None, the created figure. Otherwise the figure to which `ax` is connected. NaxlinezProbabilities of 2nd SamplezProbabilities of 1st SamplezTheoretical ProbabilitieszSample Probabilities?) r'rrLrrY_do_plotr-r^ set_xlabel set_ylabelset_xlimset_ylim) rAxlabelylabelrbotherra plotkwargs check_otherp_xecdf_xfigs rppplotzProbPlot.ppplot%sIh  $UH55K ( .C1T%011$2GHHFVTY+-D'F f f2ty$)444 r)NNNNN)NNNNNF)NNNFN)__name__ __module__ __qualname____doc__rnormrHrrLrSrWrYr^rrboolr r~rrrrrsjVVvZ  Q0Q0Q0Q0f//^/ ! !^ !,,^,$$^$((^(   YYYYz  iiiiiiZ   SSSSSSrrrFc Rt|||||||} | jd||d| } | S)ak Q-Q plot of the quantiles of x versus the quantiles/ppf of a distribution. Can take arguments specifying the parameters for dist or fit them automatically. (See fit under Parameters.) Parameters ---------- data : array_like A 1d data array. dist : callable Comparison distribution. The default is scipy.stats.distributions.norm (a standard normal). distargs : tuple A tuple of arguments passed to dist to specify it fully so dist.ppf may be called. a : float Offset for the plotting position of an expected order statistic, for example. The plotting positions are given by (i - a)/(nobs - 2*a + 1) for i in range(0,nobs+1) loc : float Location parameter for dist scale : float Scale parameter for dist fit : bool If fit is false, loc, scale, and distargs are passed to the distribution. If fit is True then the parameters for dist are fit automatically using dist.fit. The quantiles are formed from the standardized data, after subtracting the fitted loc and dividing by the fitted scale. line : {None, "45", "s", "r", "q"} Options for the reference line to which the data is compared: - "45" - 45-degree line - "s" - standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - "r" - A regression line is fit - "q" - A line is fit through the quartiles. - None - by default no reference line is added to the plot. ax : AxesSubplot, optional If given, this subplot is used to plot in instead of a new figure being created. **plotkwargs Additional matplotlib arguments to be passed to the `plot` command. Returns ------- Figure If `ax` is None, the created figure. Otherwise the figure to which `ax` is connected. See Also -------- scipy.stats.probplot Notes ----- Depends on matplotlib. If `fit` is True then the parameters are fit using the distribution's fit() method. Examples -------- >>> import statsmodels.api as sm >>> from matplotlib import pyplot as plt >>> data = sm.datasets.longley.load() >>> exog = sm.add_constant(data.exog) >>> mod_fit = sm.OLS(data.endog, exog).fit() >>> res = mod_fit.resid # residuals >>> fig = sm.qqplot(res) >>> plt.show() qqplot of the residuals against quantiles of t-distribution with 4 degrees of freedom: >>> import scipy.stats as stats >>> fig = sm.qqplot(res, stats.t, distargs=(4,)) >>> plt.show() qqplot against same as above, but with mean 3 and std 10: >>> fig = sm.qqplot(res, stats.t, distargs=(4,), loc=3, scale=10) >>> plt.show() Automatically determine parameters for t distribution including the loc and scale: >>> fig = sm.qqplot(res, stats.t, fit=True, line="45") >>> plt.show() The following plot displays some options, follow the link to see the code. .. plot:: plots/graphics_gofplots_qqplot.py )r-r r&r#rrr`r)rr ) r"r-r r#rrr&rbrarmr~rqs rr r AsMV 4(qcH (/ 9Rd 9 9j 9 9C JrcNt|tst|}t|tst|}|jjd|jjdkr||||||}n||||||d}|S)a Q-Q Plot of two samples' quantiles. Can take either two `ProbPlot` instances or two array-like objects. In the case of the latter, both inputs will be converted to `ProbPlot` instances using only the default values - so use `ProbPlot` instances if finer-grained control of the quantile computations is required. Parameters ---------- data1 : {array_like, ProbPlot} Data to plot along x axis. If the sample sizes are unequal, the longer series is always plotted along the x-axis. data2 : {array_like, ProbPlot} Data to plot along y axis. Does not need to have the same number of observations as data 1. If the sample sizes are unequal, the longer series is always plotted along the x-axis. xlabel : {None, str} User-provided labels for the x-axis. If None (default), other values are used. ylabel : {None, str} User-provided labels for the y-axis. If None (default), other values are used. line : {None, "45", "s", "r", q"} Options for the reference line to which the data is compared: - "45" - 45-degree line - "s" - standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - "r" - A regression line is fit - "q" - A line is fit through the quartiles. - None - by default no reference line is added to the plot. ax : AxesSubplot, optional If given, this subplot is used to plot in instead of a new figure being created. Returns ------- Figure If `ax` is None, the created figure. Otherwise the figure to which `ax` is connected. See Also -------- scipy.stats.probplot Notes ----- 1) Depends on matplotlib. 2) If `data1` and `data2` are not `ProbPlot` instances, instances will be created using the default parameters. Therefore, it is recommended to use `ProbPlot` instance if fine-grained control is needed in the computation of the quantiles. Examples -------- >>> import statsmodels.api as sm >>> import numpy as np >>> import matplotlib.pyplot as plt >>> from statsmodels.graphics.gofplots import qqplot_2samples >>> x = np.random.normal(loc=8.5, scale=2.5, size=37) >>> y = np.random.normal(loc=8.0, scale=3.0, size=37) >>> pp_x = sm.ProbPlot(x) >>> pp_y = sm.ProbPlot(y) >>> qqplot_2samples(pp_x, pp_y) >>> plt.show() .. plot:: plots/graphics_gofplots_qqplot_2samples.py >>> fig = qqplot_2samples(pp_x, pp_y, xlabel=None, ylabel=None, ... line=None, ax=None) r)rjrkrbrlraT)rjrkrbrlrars)r'rr"r$r )data1data2rjrkrbrarqs rr r sZ eX & &  eX & &  zUZ-a000ll&t5R  ll    Jrr-c J|}dD]4}||vr.|d|||d}n5dD]4}||vr.|d|||d}n5|r|d||dkrt||} t | d| d<t| d | d <|j| | fi|| | | | d S||td tj |}tj |}|d krEt|t|j}|j||fi|d S|d krAtj|tj|} } || z| z} |j|| fi|d S|dkrt)|dt+j|d} t+j|d}|ddg}|| z tj|z } | | |dzz } |j|| |z| zfi|d Sd S)a Plot a reference line for a qqplot. Parameters ---------- ax : matplotlib axes instance The axes on which to plot the line line : str {"45","r","s","q"} Options for the reference line to which the data is compared.: - "45" - 45-degree line - "s" - standardized line, the expected order statistics are scaled by the standard deviation of the given sample and have the mean added to them - "r" - A regression line is fit - "q" - A line is fit through the quartiles. - None - By default no reference line is added to the plot. x : ndarray X data for plot. Not needed if line is "45". y : ndarray Y data for plot. Not needed if line is "45". dist : scipy.stats.distribution A scipy.stats distribution, needed if line is "q". fmt : str, optional Line format string passed to `plot`. **lineoptions Additional arguments to be passed to the `plot` command. Notes ----- There is no return value. The line is plotted on the given `ax`. Examples -------- Import the food expenditure dataset. Plot annual food expenditure on x-axis and household income on y-axis. Use qqline to add regression line into the plot. >>> import statsmodels.api as sm >>> import numpy as np >>> import matplotlib.pyplot as plt >>> from statsmodels.graphics.gofplots import qqline >>> foodexp = sm.datasets.engel.load() >>> x = foodexp.exog >>> y = foodexp.endog >>> ax = plt.subplot(111) >>> plt.scatter(x, y) >>> ax.set_xlabel(foodexp.exog_name[0]) >>> ax.set_ylabel(foodexp.endog_name) >>> qqline(ax, "r", x, y) >>> plt.show() .. plot:: plots/graphics_gofplots_qqplot_qqline.py )-z--z-.: linestyle).rov^<>12348srzP*hH+xXDd|_markercolor45rr Nz*If line is not 45, x and y cannot be None.rrqrNKg?g?)copy setdefaultreplacerget_xlimget_ylimminmaxplotrhrir+r9rVrrr& fittedvaluesstdmeanr\rscoreatpercentilerNdiff)rarbryr-fmt lineoptionslsrend_ptsmbref_lineq25q75theoretical_quartiless rr r sr""$$K$ 99  " "; 3 3 3++b"%%C E 6 S==  " "8V 4 4 4++fb))C E  -w,,, t||r{{}}bkkmm44__ __ 00K000 G GyAIEFFF  A  A s{{ <?? # # ' ' ) ) 61$$ $$$$$ vayy"'!**1q5198++{+++++ 4%a,,%a,, $$ 6 6 3Y"'"788 8 !+A.. .1q519,, ,,,,, rrcc\||n|}tjd|dz|z |dz|z |z z S)a Generates sequence of plotting positions Parameters ---------- nobs : int Number of probability points to plot a : float, default 0.0 alpha parameter for the plotting position of an expected order statistic b : float, default None beta parameter for the plotting position of an expected order statistic. If None, then b is set to a. Returns ------- ndarray The plotting positions Notes ----- The plotting positions are given by (i - a)/(nobs + 1 - a - b) for i in range(1, nobs+1) See Also -------- scipy.stats.mstats.plotting_positions Additional information on alpha and beta Nrdr )r9arange)r%r#rs rrJrJs><YAA Ic4!8 $ $q (TAX\A-= >>rct|dtjdddt}tjgd}tj||d|ddd z f}|d kr'tj|dz |d|ddd dz z f}|d kr'tj|dz |d|ddd dz z f}|d z}||}|||d |dzDdddd| | | gdS)a Formats a theoretical quantile axis to display the corresponding probabilities on the quantiles' scale. Parameters ---------- ax : AxesSubplot, optional The axis to be formatted nobs : scalar Number of observations in the sample dist : scipy.stats.distribution A scipy.stats distribution sufficiently specified to implement its ppf() method. Returns ------- There is no return value. This operates on `ax` in place rN Z )dtype)rdrdNr2igY@c,g|]}t|Srr)rlbls rrz&_fmt_probplot_axis..s000cS000r-anchorrightcenter)rotation rotation_modehorizontalalignmentverticalalignment) r\r9linspacefloatrVr:rN set_xticksset_xticklabelsrhrr)rar-r% axis_probssmall axis_qntlss rr|r|sh&tURQe444J H[[[ ! !Euj#ddd *;;>#3#3455555rrc \ddddd}|jd i||dd} tj|\} }|d|r|j|||fd| i|n|j|||fi||r,|dvrd |z} t| t||||| | |fS) a Boiler plate plotting function for the `ppplot`, `qqplot`, and `probplot` methods of the `ProbPlot` class Parameters ---------- x : array_like X-axis data to be plotted y : array_like Y-axis data to be plotted dist : scipy.stats.distribution A scipy.stats distribution, needed if `line` is "q". line : {"45", "s", "r", "q", None}, default None Options for the reference line to which the data is compared. ax : AxesSubplot, optional If given, this subplot is used to plot in instead of a new figure being created. fmt : str, optional matplotlib-compatible formatting string for the data markers kwargs : keywords These are passed to matplotlib.plot Returns ------- fig : Figure The figure containing `ax`. ax : AxesSubplot The original axes if provided. Otherwise a new instance. rC0none)rmarkerfacecolormarkeredgecolorrwherepreg{Gz?)rrrrz!%s option for line not understood)rrr-r) updatepopr create_mpl_ax set_xmarginsteprr+r ) rrr-rbrarrkwargs plot_stylerrqrQs rreres B JJ NN7E * *E!"%%GCNN4 )1c555*55551c((Z((( . , , ,5rs************333333777777000000 = = =nnnnnnnnf     oooof;?````F}-}-}-}-B????D%6%6%6R8=8888vHHHHHHr