0Phj}4dZddlmZmZddlZddlZddlm Z m Z m Z m Z m Z ddlmZmZmZddlmZddlmZdd lmZmZmZdd lmZdd lmZmZmZd d l m!Z!ddl"m#Z#m$Z$GddeeZ%Gdde%Z&Gdde%Z'Gdde%Z(dS)z> Generalized Linear Models with Exponential Dispersion Family )IntegralRealN) HalfGammaLossHalfPoissonLossHalfSquaredErrorHalfTweedieLossHalfTweedieLossIdentity) BaseEstimatorRegressorMixin _fit_context) check_array)_openmp_effective_n_threads)HiddenInterval StrOptions)_check_optimize_result)_check_sample_weightcheck_is_fitted validate_data)LinearModelLoss)NewtonCholeskySolver NewtonSolverc &eZdZUdZeedddgdgeddheegee d ddgeeddd gdgd gd Z e e d <dddddddd dZ edddZdZdZddZfdZdZxZS)_GeneralizedLinearRegressoraRegression via a penalized Generalized Linear Model (GLM). GLMs based on a reproductive Exponential Dispersion Model (EDM) aim at fitting and predicting the mean of the target y as y_pred=h(X*w) with coefficients w. Therefore, the fit minimizes the following objective function with L2 priors as regularizer:: 1/(2*sum(s_i)) * sum(s_i * deviance(y_i, h(x_i*w)) + 1/2 * alpha * ||w||_2^2 with inverse link function h, s=sample_weight and per observation (unit) deviance deviance(y_i, h(x_i*w)). Note that for an EDM, 1/2 * deviance is the negative log-likelihood up to a constant (in w) term. The parameter ``alpha`` corresponds to the lambda parameter in glmnet. Instead of implementing the EDM family and a link function separately, we directly use the loss functions `from sklearn._loss` which have the link functions included in them for performance reasons. We pick the loss functions that implement (1/2 times) EDM deviances. Read more in the :ref:`User Guide `. .. versionadded:: 0.23 Parameters ---------- alpha : float, default=1 Constant that multiplies the penalty term and thus determines the regularization strength. ``alpha = 0`` is equivalent to unpenalized GLMs. In this case, the design matrix `X` must have full column rank (no collinearities). Values must be in the range `[0.0, inf)`. fit_intercept : bool, default=True Specifies if a constant (a.k.a. bias or intercept) should be added to the linear predictor (X @ coef + intercept). solver : {'lbfgs', 'newton-cholesky'}, default='lbfgs' Algorithm to use in the optimization problem: 'lbfgs' Calls scipy's L-BFGS-B optimizer. 'newton-cholesky' Uses Newton-Raphson steps (in arbitrary precision arithmetic equivalent to iterated reweighted least squares) with an inner Cholesky based solver. This solver is a good choice for `n_samples` >> `n_features`, especially with one-hot encoded categorical features with rare categories. Be aware that the memory usage of this solver has a quadratic dependency on `n_features` because it explicitly computes the Hessian matrix. .. versionadded:: 1.2 max_iter : int, default=100 The maximal number of iterations for the solver. Values must be in the range `[1, inf)`. tol : float, default=1e-4 Stopping criterion. For the lbfgs solver, the iteration will stop when ``max{|g_j|, j = 1, ..., d} <= tol`` where ``g_j`` is the j-th component of the gradient (derivative) of the objective function. Values must be in the range `(0.0, inf)`. warm_start : bool, default=False If set to ``True``, reuse the solution of the previous call to ``fit`` as initialization for ``coef_`` and ``intercept_``. verbose : int, default=0 For the lbfgs solver set verbose to any positive number for verbosity. Values must be in the range `[0, inf)`. Attributes ---------- coef_ : array of shape (n_features,) Estimated coefficients for the linear predictor (`X @ coef_ + intercept_`) in the GLM. intercept_ : float Intercept (a.k.a. bias) added to linear predictor. n_iter_ : int Actual number of iterations used in the solver. _base_loss : BaseLoss, default=HalfSquaredError() This is set during fit via `self._get_loss()`. A `_base_loss` contains a specific loss function as well as the link function. The loss to be minimized specifies the distributional assumption of the GLM, i.e. the distribution from the EDM. Here are some examples: ======================= ======== ========================== _base_loss Link Target Domain ======================= ======== ========================== HalfSquaredError identity y any real number HalfPoissonLoss log 0 <= y HalfGammaLoss log 0 < y HalfTweedieLoss log dependent on tweedie power HalfTweedieLossIdentity identity dependent on tweedie power ======================= ======== ========================== The link function of the GLM, i.e. mapping from linear predictor `X @ coeff + intercept` to prediction `y_pred`. For instance, with a log link, we have `y_pred = exp(X @ coeff + intercept)`. Nleftclosedbooleanlbfgsnewton-choleskyrneitherverbosealpha fit_interceptsolvermax_itertol warm_startr&_parameter_constraints?Td-C6?Frch||_||_||_||_||_||_||_dSNr')selfr(r)r*r+r,r-r&s ]/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/sklearn/linear_model/_glm/glm.py__init__z$_GeneralizedLinearRegressor.__init__s: *   $ )prefer_skip_nested_validationct|||ddgtjtjgdd\}}|jdkr tj}n2t t |j|jtj}t||dd}|t||| }|j \}}| |_ t|j |j }|j|s"t#d |j jjd |jrjt+|drZ|jr4tj|jtj|jgf}n|j}||d}nT||| }|jr6|jjtj|||d<|j} t?} |jdkr|j } tBj"#| |dd|j$d|j%dz |j&dtj'tPj)zd|||| | f} tUd| |_+| j,}n|jdkrIt[||| |j&|j$| |j%} | .|||}| j/|_+n{ta|jtbrI|||| |j&|j$| } | .|||}| j/|_+nt#d|jd |jr|d|_|d d|_nd|_||_|S)aFit a Generalized Linear Model. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : array-like of shape (n_samples,) Target values. sample_weight : array-like of shape (n_samples,), default=None Sample weights. Returns ------- self : object Fitted model. csccsrTF) accept_sparsedtype y_numeric multi_outputr#Cr=order ensure_2dNr=) base_lossr):Some value(s) of y are out of the valid range of the loss .coef_)copyweightszL-BFGS-B2r@)maxitermaxlsiprintgtolftol)methodjacoptionsargsr$)coef linear_lossl2_reg_strengthr,r+ n_threadsr&)rXrYrZr,r+r[zInvalid solver=r)2rnpfloat64float32r*minmaxr=rrshape _get_loss _base_lossrr)rEin_y_true_range ValueError __class____name__r-hasattr concatenaterHarray intercept_astypeinit_zero_coeflinkaverager(r loss_gradientscipyoptimizeminimizer+r&r,finfofloatepsrn_iter_xrsolve iteration issubclassr)r4Xy sample_weight loss_dtype n_samples n_featuresrYrXrZr[funcopt_ressols r5fitz_GeneralizedLinearRegressor.fits(   %.:rz*   1 ;' ! !JJS!'22BJ??J 3% H H H  $1TTTM ! :..**%o,   $44Q77 <O-6<<< , ? wtW55 ! "~tz28T_> >   "2hDOcrcDJJ"DODJ r7c t|t||gdtjtjgddd}||jz|jzS)aCompute the linear_predictor = `X @ coef_ + intercept_`. Note that we often use the term raw_prediction instead of linear predictor. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Samples. Returns ------- y_pred : array of shape (n_samples,) Returns predicted values of linear predictor. )r;r:cooTF)r<r=rCallow_ndreset)rrr\r]r^rHrk)r4r|s r5_linear_predictorz-_GeneralizedLinearRegressor._linear_predictorCs_    ///:rz*   4:~//r7cn||}|jj|}|S)a*Predict using GLM with feature matrix X. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Samples. Returns ------- y_pred : array of shape (n_samples,) Returns predicted values. )rrcrninverse)r4r|raw_predictiony_preds r5predictz#_GeneralizedLinearRegressor.predict^s4//22%--n== r7c<||}t||jdd}|t|||j}|j}||st d|jdtj | |d| }||||d }|j tj || }||tj ||j d |d } d ||z| |zz z S) aJCompute D^2, the percentage of deviance explained. D^2 is a generalization of the coefficient of determination R^2. R^2 uses squared error and D^2 uses the deviance of this GLM, see the :ref:`User Guide `. D^2 is defined as :math:`D^2 = 1-\frac{D(y_{true},y_{pred})}{D_{null}}`, :math:`D_{null}` is the null deviance, i.e. the deviance of a model with intercept alone, which corresponds to :math:`y_{pred} = \bar{y}`. The mean :math:`\bar{y}` is averaged by sample_weight. Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Test samples. y : array-like of shape (n_samples,) True values of target. sample_weight : array-like of shape (n_samples,), default=None Sample weights. Returns ------- score : float D^2 of self.predict(X) w.r.t. y. r@FrANrDrFrG)y_truer~rJr)rrr~r[r)rrr=rrcrdrergr\roconstant_to_optimal_zerorntilera) r4r|r}r~rrEconstantdeviancey_mean deviance_nulls r5scorez!_GeneralizedLinearRegressor.scorepscJ//22 !5SE R R R  $1QQQMO ((++ *&***  :  . .at . L L!    9)'     $$RZ=%I%I%IJJ! 76171:66'    Hx'MH,DEEEr7ct}d|j_ |}|d |j_n#tttf$rYnwxYw|S)NTg) super__sklearn_tags__ input_tagssparserbrd target_tags positive_onlyreAttributeError TypeError)r4tagsrErfs r5rz,_GeneralizedLinearRegressor.__sklearn_tags__sww''))!% ((I1:1J1J41P1P-PD  * *NI6    D  s4A$$A>=A>ctS)a This is only necessary because of the link and power arguments of the TweedieRegressor. Note that we do not need to pass sample_weight to the loss class as this is only needed to set loss.constant_hessian on which GLMs do not rely. )rr4s r5rbz%_GeneralizedLinearRegressor._get_losss !!!r7r3)rg __module__ __qualname____doc__rrrrtyperr.dict__annotations__r6r rrrrrrb __classcell__rfs@r5rrsffX(4d6:::;# J!23 4 4 F4LL Xh4???@sD;;;< k; $ $D     &\555VVV65Vp0006$IFIFIFIFV     """""""r7rcXeZdZUdZiejZeed<ddddddd d fd Zd Z xZ S) PoissonRegressora Generalized Linear Model with a Poisson distribution. This regressor uses the 'log' link function. Read more in the :ref:`User Guide `. .. versionadded:: 0.23 Parameters ---------- alpha : float, default=1 Constant that multiplies the L2 penalty term and determines the regularization strength. ``alpha = 0`` is equivalent to unpenalized GLMs. In this case, the design matrix `X` must have full column rank (no collinearities). Values of `alpha` must be in the range `[0.0, inf)`. fit_intercept : bool, default=True Specifies if a constant (a.k.a. bias or intercept) should be added to the linear predictor (`X @ coef + intercept`). solver : {'lbfgs', 'newton-cholesky'}, default='lbfgs' Algorithm to use in the optimization problem: 'lbfgs' Calls scipy's L-BFGS-B optimizer. 'newton-cholesky' Uses Newton-Raphson steps (in arbitrary precision arithmetic equivalent to iterated reweighted least squares) with an inner Cholesky based solver. This solver is a good choice for `n_samples` >> `n_features`, especially with one-hot encoded categorical features with rare categories. Be aware that the memory usage of this solver has a quadratic dependency on `n_features` because it explicitly computes the Hessian matrix. .. versionadded:: 1.2 max_iter : int, default=100 The maximal number of iterations for the solver. Values must be in the range `[1, inf)`. tol : float, default=1e-4 Stopping criterion. For the lbfgs solver, the iteration will stop when ``max{|g_j|, j = 1, ..., d} <= tol`` where ``g_j`` is the j-th component of the gradient (derivative) of the objective function. Values must be in the range `(0.0, inf)`. warm_start : bool, default=False If set to ``True``, reuse the solution of the previous call to ``fit`` as initialization for ``coef_`` and ``intercept_`` . verbose : int, default=0 For the lbfgs solver set verbose to any positive number for verbosity. Values must be in the range `[0, inf)`. Attributes ---------- coef_ : array of shape (n_features,) Estimated coefficients for the linear predictor (`X @ coef_ + intercept_`) in the GLM. intercept_ : float Intercept (a.k.a. bias) added to linear predictor. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 n_iter_ : int Actual number of iterations used in the solver. See Also -------- TweedieRegressor : Generalized Linear Model with a Tweedie distribution. Examples -------- >>> from sklearn import linear_model >>> clf = linear_model.PoissonRegressor() >>> X = [[1, 2], [2, 3], [3, 4], [4, 3]] >>> y = [12, 17, 22, 21] >>> clf.fit(X, y) PoissonRegressor() >>> clf.score(X, y) np.float64(0.990...) >>> clf.coef_ array([0.121..., 0.158...]) >>> clf.intercept_ np.float64(2.088...) >>> clf.predict([[1, 1], [3, 4]]) array([10.676..., 21.875...]) r.r/Tr#r0r1Frr'c Xt|||||||dSNr'rr6 r4r(r)r*r+r,r-r&rfs r5r6zPoissonRegressor.__init__>E '!      r7ctSr3)rrs r5rbzPoissonRegressor._get_lossSs   r7 rgrrrrr.rrr6rbrrs@r5rrsccJ$ % <$D        *!!!!!!!r7rcXeZdZUdZiejZeed<ddddddd d fd Zd Z xZ S) GammaRegressora Generalized Linear Model with a Gamma distribution. This regressor uses the 'log' link function. Read more in the :ref:`User Guide `. .. versionadded:: 0.23 Parameters ---------- alpha : float, default=1 Constant that multiplies the L2 penalty term and determines the regularization strength. ``alpha = 0`` is equivalent to unpenalized GLMs. In this case, the design matrix `X` must have full column rank (no collinearities). Values of `alpha` must be in the range `[0.0, inf)`. fit_intercept : bool, default=True Specifies if a constant (a.k.a. bias or intercept) should be added to the linear predictor `X @ coef_ + intercept_`. solver : {'lbfgs', 'newton-cholesky'}, default='lbfgs' Algorithm to use in the optimization problem: 'lbfgs' Calls scipy's L-BFGS-B optimizer. 'newton-cholesky' Uses Newton-Raphson steps (in arbitrary precision arithmetic equivalent to iterated reweighted least squares) with an inner Cholesky based solver. This solver is a good choice for `n_samples` >> `n_features`, especially with one-hot encoded categorical features with rare categories. Be aware that the memory usage of this solver has a quadratic dependency on `n_features` because it explicitly computes the Hessian matrix. .. versionadded:: 1.2 max_iter : int, default=100 The maximal number of iterations for the solver. Values must be in the range `[1, inf)`. tol : float, default=1e-4 Stopping criterion. For the lbfgs solver, the iteration will stop when ``max{|g_j|, j = 1, ..., d} <= tol`` where ``g_j`` is the j-th component of the gradient (derivative) of the objective function. Values must be in the range `(0.0, inf)`. warm_start : bool, default=False If set to ``True``, reuse the solution of the previous call to ``fit`` as initialization for `coef_` and `intercept_`. verbose : int, default=0 For the lbfgs solver set verbose to any positive number for verbosity. Values must be in the range `[0, inf)`. Attributes ---------- coef_ : array of shape (n_features,) Estimated coefficients for the linear predictor (`X @ coef_ + intercept_`) in the GLM. intercept_ : float Intercept (a.k.a. bias) added to linear predictor. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 n_iter_ : int Actual number of iterations used in the solver. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PoissonRegressor : Generalized Linear Model with a Poisson distribution. TweedieRegressor : Generalized Linear Model with a Tweedie distribution. Examples -------- >>> from sklearn import linear_model >>> clf = linear_model.GammaRegressor() >>> X = [[1, 2], [2, 3], [3, 4], [4, 3]] >>> y = [19, 26, 33, 30] >>> clf.fit(X, y) GammaRegressor() >>> clf.score(X, y) np.float64(0.773...) >>> clf.coef_ array([0.072..., 0.066...]) >>> clf.intercept_ np.float64(2.896...) >>> clf.predict([[1, 0], [2, 8]]) array([19.483..., 35.795...]) r.r/Tr#r0r1Frr'c Xt|||||||dSrrrs r5r6zGammaRegressor.__init__rr7ctSr3)rrs r5rbzGammaRegressor._get_losss r7rrs@r5rrWsddL$ % <$D        *r7rc eZdZUdZiejeedddgehdgdZe e d<dd d d d d dddd fd Z dZ xZ S)TweedieRegressora:Generalized Linear Model with a Tweedie distribution. This estimator can be used to model different GLMs depending on the ``power`` parameter, which determines the underlying distribution. Read more in the :ref:`User Guide `. .. versionadded:: 0.23 Parameters ---------- power : float, default=0 The power determines the underlying target distribution according to the following table: +-------+------------------------+ | Power | Distribution | +=======+========================+ | 0 | Normal | +-------+------------------------+ | 1 | Poisson | +-------+------------------------+ | (1,2) | Compound Poisson Gamma | +-------+------------------------+ | 2 | Gamma | +-------+------------------------+ | 3 | Inverse Gaussian | +-------+------------------------+ For ``0 < power < 1``, no distribution exists. alpha : float, default=1 Constant that multiplies the L2 penalty term and determines the regularization strength. ``alpha = 0`` is equivalent to unpenalized GLMs. In this case, the design matrix `X` must have full column rank (no collinearities). Values of `alpha` must be in the range `[0.0, inf)`. fit_intercept : bool, default=True Specifies if a constant (a.k.a. bias or intercept) should be added to the linear predictor (`X @ coef + intercept`). link : {'auto', 'identity', 'log'}, default='auto' The link function of the GLM, i.e. mapping from linear predictor `X @ coeff + intercept` to prediction `y_pred`. Option 'auto' sets the link depending on the chosen `power` parameter as follows: - 'identity' for ``power <= 0``, e.g. for the Normal distribution - 'log' for ``power > 0``, e.g. for Poisson, Gamma and Inverse Gaussian distributions solver : {'lbfgs', 'newton-cholesky'}, default='lbfgs' Algorithm to use in the optimization problem: 'lbfgs' Calls scipy's L-BFGS-B optimizer. 'newton-cholesky' Uses Newton-Raphson steps (in arbitrary precision arithmetic equivalent to iterated reweighted least squares) with an inner Cholesky based solver. This solver is a good choice for `n_samples` >> `n_features`, especially with one-hot encoded categorical features with rare categories. Be aware that the memory usage of this solver has a quadratic dependency on `n_features` because it explicitly computes the Hessian matrix. .. versionadded:: 1.2 max_iter : int, default=100 The maximal number of iterations for the solver. Values must be in the range `[1, inf)`. tol : float, default=1e-4 Stopping criterion. For the lbfgs solver, the iteration will stop when ``max{|g_j|, j = 1, ..., d} <= tol`` where ``g_j`` is the j-th component of the gradient (derivative) of the objective function. Values must be in the range `(0.0, inf)`. warm_start : bool, default=False If set to ``True``, reuse the solution of the previous call to ``fit`` as initialization for ``coef_`` and ``intercept_`` . verbose : int, default=0 For the lbfgs solver set verbose to any positive number for verbosity. Values must be in the range `[0, inf)`. Attributes ---------- coef_ : array of shape (n_features,) Estimated coefficients for the linear predictor (`X @ coef_ + intercept_`) in the GLM. intercept_ : float Intercept (a.k.a. bias) added to linear predictor. n_iter_ : int Actual number of iterations used in the solver. n_features_in_ : int Number of features seen during :term:`fit`. .. versionadded:: 0.24 feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PoissonRegressor : Generalized Linear Model with a Poisson distribution. GammaRegressor : Generalized Linear Model with a Gamma distribution. Examples -------- >>> from sklearn import linear_model >>> clf = linear_model.TweedieRegressor() >>> X = [[1, 2], [2, 3], [3, 4], [4, 3]] >>> y = [2, 3.5, 5, 5.5] >>> clf.fit(X, y) TweedieRegressor() >>> clf.score(X, y) np.float64(0.839...) >>> clf.coef_ array([0.599..., 0.299...]) >>> clf.intercept_ np.float64(1.600...) >>> clf.predict([[1, 1], [3, 4]]) array([2.500..., 4.599...]) Nr%r >logautoidentity)powerrnr.rr/Trr#r0r1Fr) rr(r)rnr*r+r,r-r&c tt||||||| ||_||_dSr)rr6rnr) r4rr(r)rnr*r+r,r-r&rfs r5r6zTweedieRegressor.__init__fsO '!      r7c|jdkr5|jdkrt|jSt|jS|jdkrt|jS|jdkrt|jSdS)Nrr)rrr)rnrr r rs r5rbzTweedieRegressor._get_losss 9  zQ.TZ@@@@'TZ8888 9  "444 4 9 " "*<<< < # "r7)rgrrrrr.rrrrrr6rbrrs@r5rrsBBH$ % <$(4tI>>>?777889$$$D  2 = = = = = = =r7r))rnumbersrrnumpyr\scipy.optimizerq _loss.lossrrrr r baser r r utilsrutils._openmp_helpersrutils._param_validationrrrutils.optimizerutils.validationrrr _linear_lossr_newton_solverrrrrrrr7r5rs #"""""""@????????? @@@@@@CCCCCCCCCC444444TTTTTTTTTT******>>>>>>>>s"s"s"s"s".-s"s"s"l @!@!@!@!@!2@!@!@!FAAAAA0AAAHq=q=q=q=q=2q=q=q=q=q=r7