0PhGJdZddlZddlmZddlmZdZGddZdS) zA Loss functions for linear models with raw_prediction = X @ coef N)sparse) squared_normc|jd}tj|r7|jtj|df||fz|zS|dddf|z}|j|zS)z/Compute the sandwich product X.T @ diag(W) @ X.rshapeN)rrissparseT dia_matrixtoarray)XW n_samplesWXs a/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/sklearn/linear_model/_linear_loss.py sandwich_dotrsy I q C&#QF9i2HIII IA M '))  qqq$wZ!^sRxceZdZdZdZddZdZdZdZ dd Z dd Z dd Z dd Z ddZ dS)LinearModelLossa General class for loss functions with raw_prediction = X @ coef + intercept. Note that raw_prediction is also known as linear predictor. The loss is the average of per sample losses and includes a term for L2 regularization:: loss = 1 / s_sum * sum_i s_i loss(y_i, X_i @ coef + intercept) + 1/2 * l2_reg_strength * ||coef||_2^2 with sample weights s_i=1 if sample_weight=None and s_sum=sum_i s_i. Gradient and hessian, for simplicity without intercept, are:: gradient = 1 / s_sum * X.T @ loss.gradient + l2_reg_strength * coef hessian = 1 / s_sum * X.T @ diag(loss.hessian) @ X + l2_reg_strength * identity Conventions: if fit_intercept: n_dof = n_features + 1 else: n_dof = n_features if base_loss.is_multiclass: coef.shape = (n_classes, n_dof) or ravelled (n_classes * n_dof,) else: coef.shape = (n_dof,) The intercept term is at the end of the coef array: if base_loss.is_multiclass: if coef.shape (n_classes, n_dof): intercept = coef[:, -1] if coef.shape (n_classes * n_dof,) intercept = coef[n_features::n_dof] = coef[(n_dof-1)::n_dof] intercept.shape = (n_classes,) else: intercept = coef[-1] Shape of gradient follows shape of coef. gradient.shape = coef.shape But hessian (to make our lives simpler) are always 2-d: if base_loss.is_multiclass: hessian.shape = (n_classes * n_dof, n_classes * n_dof) else: hessian.shape = (n_dof, n_dof) Note: If coef has shape (n_classes * n_dof,), the 2d-array can be reconstructed as coef.reshape((n_classes, -1), order="F") The option order="F" makes coef[:, i] contiguous. This, in turn, makes the coefficients without intercept, coef[:, :-1], contiguous and speeds up matrix-vector computations. Note: If the average loss per sample is wanted instead of the sum of the loss per sample, one can simply use a rescaled sample_weight such that sum(sample_weight) = 1. Parameters ---------- base_loss : instance of class BaseLoss from sklearn._loss. fit_intercept : bool c"||_||_dSN) base_loss fit_intercept)selfrrs r__init__zLinearModelLoss.__init__hs"*rNc|jd}|jj}|jr|dz}n|}|jjrt j|||f|d}nt j|||}|S)aAllocate coef of correct shape with zeros. Parameters: ----------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. dtype : data-type, default=None Overrides the data type of coef. With dtype=None, coef will have the same dtype as X. Returns ------- coef : ndarray of shape (n_dof,) or (n_classes, n_dof) Coefficients of a linear model. F)rdtypeorder)rr)rr n_classesr is_multiclassnp zeros_like)rr r n_featuresr!n_dofcoefs rinit_zero_coefzLinearModelLoss.init_zero_coefls| WQZ N,   NEEE > ' >=9e*11rcP|jdkr||znt|}d|z|zS)z5Compute L2 penalty term l2_reg_strength/2 *||w||_2^2.rg?)r-r)rr0l2_reg_strengthnorm2_ws r l2_penaltyzLinearModelLoss.l2_penaltys5'.|q'8'8'G##l7>S>S_$w..rr+rc||||\}} }n||\}} |j||d|} t j| |} | |||zS)aCompute the loss as weighted average over point-wise losses. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- loss : float Weighted average of losses per sample, plus penalty. Ny_truer3 sample_weight n_threadsr0)r4r1rlossr#averager8) rr'r yr<r6r=r3r0r/r?s rr?zLinearModelLoss.losssN  !151J1J4QR1S1S .GY!%!6!6t!|j jrdnd}|d|d| | z| dzxx|z cc<|jr3|j|z}||dddf<||dddf<||d<nt|j|| || \}}||z}|| | fd}|j|z||zz|ddd| f<|jr|d |dddf<|jdkr|d}|||z }nd|z }t7| D]j}|dd|fd|dd|fz z|z}t-||||| | z| || | z| f<|jrU|j|z}|||| | z| | | z|zf<||| | z|z|| | z| f<||| | z|z| | z|zf<t7|dz| D]}|dd|f |dd|fz|z}t-||||| | z| || | z| f<|jrU|j|z}|||| | z| | | z|zf<||| | z|z|| | z| f<||| | z|z| | z|zf<||d| |d| f||d| |d| f<l|d krG|j jrdnd}|d|d| dz| z| z| | zdzxx|z cc<d}|||fS)a~Computes gradient and hessian w.r.t. coef. Parameters ---------- coef : ndarray of shape (n_dof,), (n_classes, n_dof) or (n_classes * n_dof,) Coefficients of a linear model. If shape (n_classes * n_dof,), the classes of one feature are contiguous, i.e. one reconstructs the 2d-array via coef.reshape((n_classes, -1), order="F"). X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : contiguous array of shape (n_samples,) Observed, true target values. sample_weight : None or contiguous array of shape (n_samples,), default=None Sample weights. l2_reg_strength : float, default=0.0 L2 regularization strength n_threads : int, default=1 Number of OpenMP threads to use. gradient_out : None or ndarray of shape coef.shape A location into which the gradient is stored. If None, a new array might be created. hessian_out : None or ndarray of shape (n_dof, n_dof) or (n_classes * n_dof, n_classes * n_dof) A location into which the hessian is stored. If None, a new array might be created. raw_prediction : C-contiguous array of shape (n_samples,) or array of shape (n_samples, n_classes) Raw prediction values (in link space). If provided, these are used. If None, then raw_prediction = X @ coef + intercept is calculated. Returns ------- gradient : ndarray of shape coef.shape The gradient of the loss. hessian : ndarray of shape (n_dof, n_dof) or (n_classes, n_dof, n_dof, n_classes) Hessian matrix. hessian_warning : bool True if pointwise hessian has more than 25% of its elements non-positive. NrrDz4gradient_out is required to have shape coef.shape = z; got .z"gradient_out must be F-contiguous.rCz'hessian_out is required to have shape (z); got hessian_out.shape=zhessian_out must be contiguous.r:rr>g?r*Cr,r)r*r*rEg?rF)rrr!rGrr4r1r#rIrJr ValueErrorr"flags f_contiguoussizerK c_contiguousgradient_hessianr@absr rr.gradient_probar-rLrange)rr'r rAr<r6r= gradient_out hessian_outr3rr%r!r&r0r/rNrOnhessrMhess_pointwisehessian_warningr Xhprobaswkhls rrZz LinearModelLoss.gradient_hessiansn./Wdn6N*JS!3444  !151J1J4QR1S1S .GY!%!6!6t!?? ?D~+b $-1^-L-L-+# .M.. *NN f $N f $N  >Q. FFFM  VN33N !n 47P PD* ! 0)--//R 3T?22-9!^-L-LD*kzk) *"" $z6?C Ru --8zE)eai8$%! 4S>) "SbS"W "R"W -1133V %)N$A$A-+# %B%% !NE f $N<<E 2#<>>D#1#3a#7/G:S#SDKZK ! 9,00a088QQQU yA~~zzz,,L("V+6\9%%, X, X!!!Q$K1uQQQT{?3b8!A&& J.: J.:<% qBI 2Y>!J.24!J.2I 2Y>@ Z/!3Y5Ka5OOPq1ui00XXAqqq!t uQQQT{2R7A%Q**I 2Y>I 2Y>@) S1W J 6B% 2Q68% 2Q6 J 6BD EEGGY3a7Z9ORS9SST8.hesspsmA&&?1%%V'(sb1[j[>.A'BC  $$')y':':ACQ{ {^;T'U'UC  $KZK   Oa  n$DD   %L  $$$"6$$$$q*~5 ae8KKCG rrrDcB |dfd} jr|dddf}|ddddf}nd}|jz|z}| |zdddtjfz }| z} | ddtjfz}t jfjd}|jz z |zz|ddd f< jr |d z |dddf<jdkr| dS|S)Nr*rr,rrrErD) r.rr rIr#newaxisrKrr-rL)rn s_intercepttmp hess_prodr r'r6r!r&r%rer<rrNr0s rrsz7LinearModelLoss.gradient_hessian_product..hessps\IIy"oSI99%$"#AAArE(K!!!SbS& AA"#K!#g + ))q)11!!!RZ-@@u  ,=BJ77CHi%7w}TWXXX -0UQY&,@?UVCV,V !!![j[.)%@'*wwAw'?Iaaae$9>>$???555$$rrr,)rrr!rGrr4r#rIr"rZrJrr rr r rusqueezeasarray atleast_1dr\rKr-rL)rr'r rAr<r6r=rr/r3rMrbrOrsrprqrrr!r&r%rerNr0s``` `` @@@@@@@@@rgradient_hessian_productz(LinearModelLoss.gradient_hessian_products-@./Wdn6N*JS!3444-1-F-FtQ-O-O*N+3 9N9N~+p 4-1^-L-L-+# .M.. *NN f $N f $N=W];;;D !n 47P PD* ! 0)--//R),,..Kq!! 7%~q&9)YAWXXX $AAArzM2Q6! /BJrvv1v~~$>$>??v..            &%)N$A$A-+# %B%% !NE f $N8Y.gm3OOOD#1#3a#7/G:S#SDKZK ! 9,00a088QQQU . % % % % % % % % % % % % % % %.yA~~zzz,,e33U{rr)Nr+rN)Nr+rNNN)Nr+r)__name__ __module__ __qualname____doc__rr(r1r4r8r?rHrQrZr|rrrr%s6@@D+++8%"%"%"N222@///4@4@4@4@vLLLLfGGGG\++++DNOWWWWWWrr) rnumpyr#scipyr utils.extmathrrrrrrrs((((((.T T T T T T T T T T r