M/Ph(PdZddlZddlmZddZdZdZd Zd Z d Z d Z d Z dS)ai Univariate lowess function, like in R. References ---------- Hastie, Tibshirani, Friedman. (2009) The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition: Chapter 6. Cleveland, W.S. (1979) "Robust Locally Weighted Regression and Smoothing Scatterplots". Journal of the American Statistical Association 74 (368): 829-836. N)lstsqUUUUUU?c X|}|jdkrtd|jdkrtd|jd|jdkrtd|jd}tj|}t ||z}tj|}tj||} ||} t| | ||\}} t|D]} t| | || ||tj| |gj } |df| _| S)a LOWESS (Locally Weighted Scatterplot Smoothing) A lowess function that outs smoothed estimates of endog at the given exog values from points (exog, endog) Parameters ---------- endog : 1-D numpy array The y-values of the observed points exog : 1-D numpy array The x-values of the observed points frac : float Between 0 and 1. The fraction of the data used when estimating each y-value. it : int The number of residual-based reweightings to perform. Returns ------- out: numpy array A numpy array with two columns. The first column is the sorted x values and the second column the associated estimated y-values. Notes ----- This lowess function implements the algorithm given in the reference below using local linear estimates. Suppose the input data has N points. The algorithm works by estimating the true ``y_i`` by taking the frac*N closest points to ``(x_i,y_i)`` based on their x values and estimating ``y_i`` using a weighted linear regression. The weight for ``(x_j,y_j)`` is `_lowess_tricube` function applied to ``|x_i-x_j|``. If ``iter > 0``, then further weighted local linear regressions are performed, where the weights are the same as above times the `_lowess_bisquare` function of the residuals. Each iteration takes approximately the same amount of time as the original fit, so these iterations are expensive. They are most useful when the noise has extremely heavy tails, such as Cauchy noise. Noise with less heavy-tails, such as t-distributions with ``df > 2``, are less problematic. The weights downgrade the influence of points with large residuals. In the extreme case, points whose residuals are larger than 6 times the median absolute residual are given weight 0. Some experimentation is likely required to find a good choice of frac and iter for a particular dataset. References ---------- Cleveland, W.S. (1979) "Robust Locally Weighted Regression and Smoothing Scatterplots". Journal of the American Statistical Association 74 (368): 829-836. Examples -------- The below allows a comparison between how different the fits from `lowess` for different values of frac can be. >>> import numpy as np >>> import statsmodels.api as sm >>> lowess = sm.nonparametric.lowess >>> x = np.random.uniform(low=-2*np.pi, high=2*np.pi, size=500) >>> y = np.sin(x) + np.random.normal(size=len(x)) >>> z = lowess(y, x) >>> w = lowess(y, x, frac=1./3) This gives a similar comparison for when it is 0 vs not. >>> import scipy.stats as stats >>> x = np.random.uniform(low=-2*np.pi, high=2*np.pi, size=500) >>> y = np.sin(x) + stats.cauchy.rvs(size=len(x)) >>> z = lowess(y, x, frac= 1./3, it=0) >>> w = lowess(y, x, frac=1./3) zexog must be a vectorzendog must be a vectorrz$exog and endog must have same length) ndim ValueErrorshapenpzerosintargsortarray_lowess_initial_fitrange_lowess_robustify_fitT)endogexogfracitxnfittedk index_arrayx_copyy_copyweightsiouts n/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/nonparametric/smoothers_lowess_old.pylowessr$s1` A yA~~0111 zQ1222 {1~##?@@@ 1 A Xa[[F D1H A*T""K Xd;' ( (F ; F)&&!Q??OFG 2YY--fff%q! - - - - (FF# $ $ &C1CI Jctj||f|j}d|g}tj|df}tj|}t |D]b}||||dz } ||ddz ||z } t | | } t ||ddf||d|d||| t||ddftj||ddf||ddf<||d|d|dddf<||ddf||d|dz} t||ddf |d|z| dd} | d| d||zz||<t|||dzd||fS)a The initial weighted local linear regression for lowess. Parameters ---------- x_copy : 1-d ndarray The x-values/exogenous part of the data being smoothed y_copy : 1-d ndarray The y-values/ endogenous part of the data being smoothed k : int The number of data points which affect the linear fit for each estimated point n : int The total number of points Returns ------- fitted : 1-d ndarray The fitted y-values weights : 2-d ndarray An n by k array. The contribution to the weights in the local linear fit coming from the distances between the x-values )dtyperrrNrcond) r r r'onesrmax_lowess_wt_standardize_lowess_tricubesqrtrreshape_lowess_update_nn)rrrrr nn_indicesXrr! left_width right_widthwidthy_ibetas r#rr|s4h!ufl333GAJ 1A Xa[[F 1XX33AY 1 !66 Z]1_-q 9 J ,,wqs| &z!}Z]'B C"1Iu . . . !!! %%%wwqs|,,!!!  1 jm34!!!A#aclVJqM*Q-$?@@WQqqqS\))!A..2CrBBB1EGd1gfQi//q &*ac2222 7?r%c(||dd<||z}||z}dS)a. The initial phase of creating the weights. Subtract the current x_i and divide by the width. Parameters ---------- weights : ndarray The memory where (new_entries - x_copy_i)/width will be placed new_entries : ndarray The x-values of the k closest points to x[i] x_copy_i : float x[i], the i'th point in the (sorted) x values width : float The maximum distance between x[i] and any point in new_entries Returns ------- Nothing. The modifications are made to weight in place. N)r new_entriesx_copy_ir6s r#r-r-s)(GAAAJ xG uGGGr%cd|g}tj|df}tj|}|f|_||z}tj|}tj|} |d| zz}|dk} t |d|| <t|D]} || ddftj||d|dz} ||d|d|dddf<| ||d|dz} |df| _t| |z| dd}|d|d|| zz|| <t||| dzdS)ah Additional weighted local linear regressions, performed if iter>0. They take into account the sizes of the residuals, to eliminate the effect of extreme outliers. Parameters ---------- x_copy : 1-d ndarray The x-values/exogenous part of the data being smoothed y_copy : 1-d ndarray The y-values/ endogenous part of the data being smoothed fitted : 1-d ndarray The fitted y-values from the previous iteration weights : 2-d ndarray An n by k array. The contribution to the weights in the local linear fit coming from the distances between the x-values k : int The number of data points which affect the linear fit for each estimated point n : int The total number of points Returns ------- Nothing. The fitted values are modified in place. rrrNr(r)) r r+copyr absolutemedian_lowess_bisquarerr/rr1)rrrr rrr2r3residual_weightsstoo_bigr! total_weightsr7r8s r#rrs8AJ 1AwvT{#344 "##A1!G%&&& !W1XX 3 3!!! rw/? 1 8B1 AF0G(H(HH  1 jm34!!!A#fZ]:a=%@AA e ]Q&2666q9Gd1gq 11q &*ac2222 3 3r%c |d|jkrS||||dz }||d||z }||kr|ddz|d<|ddz|d<ndSdSg)a Update the endpoints of the nearest neighbors to the ith point. Parameters ---------- x : iterable The sorted points of x-values cur_nn : list of length 2 The two current indices between which are the k closest points to x[i]. (The actual value of k is irrelevant for the algorithm. i : int The index of the current value in x for which the k closest points are desired. Returns ------- Nothing. It modifies cur_nn in place. TrrN)size)rcur_nnr! left_distnew_right_dists r#r1r1s*  !9QV  !q|+Ivay\AaD0N ))"1IMq "1IMq  E r%ctj||dd<t|tj||dd<|dz }t|dS)a The _tricube function applied to a numpy array. The tricube function is (1-abs(t)**3)**3. Parameters ---------- t : ndarray Array the tricube function is applied to elementwise and in-place. Returns ------- Nothing Nr)r r@_lowess_mycubenegativets r#r.r.'sY ;q>>AaaaD1 ;q>>AaaaDFA1r%c||z}||z}dS)z Fast matrix cube Parameters ---------- t : ndarray Array that is cubed, elementwise and in-place Returns ------- Nothing Nr:)rPt2s r#rMrM>s 1BGAAAr%cV||z}tj||dd<|dz }||z}dS)a The bisquare function applied to a numpy array. The bisquare function is (1-t**2)**2. Parameters ---------- t : ndarray array bisquare function is applied to, element-wise and in-place. Returns ------- Nothing Nr)r rNrOs r#rBrBPs8FA ;q>>AaaaDFAFAAAr%)rr) __doc__numpyr numpy.linalgrr$rr-rr1r.rMrBr:r%r#rWskkkk\444n2636363rD.   $     r%