^MhO6dZddlmZddlZddlmZddlmZm Z m Z ddl m Z ddl mZmZejejZdZ d&d Zd Zd Zd'dZd(dZd)dZdZdZd*dZd*dZdZdZ dZ!dZ"dZ#dZ$dZ%d)dZ&dZ'dZ(d Z)d+d"Z*d+d#Z+d$Z,d%Z-dS),z+Functions used by least-squares algorithms.)copysignN)norm) cho_factor cho_solve LinAlgError)issparse)LinearOperatoraslinearoperatorcrtj||}|dkrtdtj||}tj|||dzz }|dkrtdtj||z||zz }|t ||z }||z }||z } || kr|| fS| |fS)aqFind the intersection of a line with the boundary of a trust region. This function solves the quadratic equation with respect to t ||(x + s*t)||**2 = Delta**2. Returns ------- t_neg, t_pos : tuple of float Negative and positive roots. Raises ------ ValueError If `s` is zero or `x` is not within the trust region. rz `s` is zero.z#`x` is not within the trust region.)npdot ValueErrorsqrtr) xsDeltaabcdqt1t2s Z/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/scipy/optimize/_lsq/common.pyintersect_trust_regionrs q! AAvv((( q! A q! uaxA1uu>??? !ac A hq!nn A QB QB Bww2v 2v {Gz? c d} ||z} ||kr t|z|dz} |d| k} nd} | r1|||z  } t| |kr| ddfSt| |z }| r| d| ||\}}| |z }nd}|| s |dkrtd|z||zdz}n|}t |D]}||ks||krtd|z||zdz}| || ||\}}|dkr|}||z }t|||z }|||z|z|z z}t j|||zkrn|| |d z|zz  } | |t| z z} | ||d zfS) aSolve a trust-region problem arising in least-squares minimization. This function implements a method described by J. J. More [1]_ and used in MINPACK, but it relies on a single SVD of Jacobian instead of series of Cholesky decompositions. Before running this function, compute: ``U, s, VT = svd(J, full_matrices=False)``. Parameters ---------- n : int Number of variables. m : int Number of residuals. uf : ndarray Computed as U.T.dot(f). s : ndarray Singular values of J. V : ndarray Transpose of VT. Delta : float Radius of a trust region. initial_alpha : float, optional Initial guess for alpha, which might be available from a previous iteration. If None, determined automatically. rtol : float, optional Stopping tolerance for the root-finding procedure. Namely, the solution ``p`` will satisfy ``abs(norm(p) - Delta) < rtol * Delta``. max_iter : int, optional Maximum allowed number of iterations for the root-finding procedure. Returns ------- p : ndarray, shape (n,) Found solution of a trust-region problem. alpha : float Positive value such that (J.T*J + alpha*I)*p = -J.T*f. Sometimes called Levenberg-Marquardt parameter. n_iter : int Number of iterations made by root-finding procedure. Zero means that Gauss-Newton step was selected as the solution. References ---------- .. [1] More, J. J., "The Levenberg-Marquardt Algorithm: Implementation and Theory," Numerical Analysis, ed. G. A. Watson, Lecture Notes in Mathematics 630, Springer Verlag, pp. 105-116, 1977. c|dz|z}t||z }||z }tj|dz|dzz  |z }||fS)zFunction of which to find zero. It is defined as "norm of regularized (by alpha) least-squares solution minus `Delta`". Refer to [1]_. r )rr sum)alphasufrrdenomp_normphi phi_primes rphi_and_derivativez2solve_lsq_trust_region..phi_and_derivativejsY 1u cEk""unVC1Huax/00069 I~rrFgNgMbP??r )EPSrrmaxranger abs)nmufrVr initial_alphartolmax_iterr*r% threshold full_rankp alpha_upperr(r) alpha_lowerr$itratios rsolve_lsq_trust_regionr@9sb    b&C Avv!GadN bEI%   UU26]]N 77e  c19 s))e#K++Ca??YdY&  I-12D2DEK'+ *Cc)IJJHoo ;  %+"5"5 +kK.G#-MNNE++E35AAY 77Ki+uu}55  #+&.. 6#;; % % E & sadUl# $ $$A aA eR!V rcX t|\}}t||f| }tj|||dzkr|dfSn#t$rYnwxYw|d|dzz}|d|dzz}|d|dzz}|d|z} |d|z} tj| | zd||z | zzd|zd| |z| zz| | z g} tj| } tj| tj| } |tj d| zd| dzzz d| dzz d| dzzz fz}d tj |||zd ztj||z} tj | }|d d |f}|d fS) azSolve a general trust-region problem in 2 dimensions. The problem is reformulated as a 4th order algebraic equation, the solution of which is found by numpy.roots. Parameters ---------- B : ndarray, shape (2, 2) Symmetric matrix, defines a quadratic term of the function. g : ndarray, shape (2,) Defines a linear term of the function. Delta : float Radius of a trust region. Returns ------- p : ndarray, shape (2,) Found solution. newton_step : bool Whether the returned solution is the Newton step which lies within the trust region. r T)rr)rr-)r-r-rr-r,axisNF) rrr rrarrayrootsrealisrealvstackr#argmin)BgrRlowerr;rrrrfcoeffstvalueis rsolve_trust_region_2drTs. a==5 5z1 % % % 6!Q<<5!8 # #d7N $       $%(A $%(A $%(A !u A !u A X aa!eai!a%qb1fqj)9A26BDDF A ")A,,  A  1q5A1H-AqDQAX/FGHHHA "&QUU1XXA... .1 =E %A !!!Q$A e8OsAA AAc~|dkr||z }n||cxkrdkrnnd}nd}|dkrd|z}n |dkr|r|dz}||fS)zUpdate the radius of a trust region based on the cost reduction. Returns ------- Delta : float New radius. ratio : float Ratio between actual and predicted reductions. rr-?g?g@)ractual_reductionpredicted_reduction step_norm bound_hitr?s rupdate_tr_radiusr\sQ #66  0 5 5 5 5A 5 5 5 5 5 t||y  )   %<rc||}tj||}||tj||z|z }|dz}tj||}|||}|tj||z }dtj||ztj||z} |9|tj||z|z }| dtj||z|zz } ||| fS||fS)aParameterize a multivariate quadratic function along a line. The resulting univariate quadratic function is given as follows:: f(t) = 0.5 * (s0 + s*t).T * (J.T*J + diag) * (s0 + s*t) + g.T * (s0 + s*t) Parameters ---------- J : ndarray, sparse matrix or LinearOperator shape (m, n) Jacobian matrix, affects the quadratic term. g : ndarray, shape (n,) Gradient, defines the linear term. s : ndarray, shape (n,) Direction vector of a line. diag : None or ndarray with shape (n,), optional Addition diagonal part, affects the quadratic term. If None, assumed to be 0. s0 : None or ndarray with shape (n,), optional Initial point. If None, assumed to be 0. Returns ------- a : float Coefficient for t**2. b : float Coefficient for t. c : float Free term. Returned only if `s0` is provided. Nr,)rr ) JrLrdiags0vrrurs rbuild_quadratic_1drcs> aA q! A  RVAHa  HA q! A ~ EE"II RVAq\\ "&A,, 2 .   T 1%% %A rvb4i,,, ,A!Qw!t rc||g}|dkr-d|z|z }||cxkr|krnn||tj|}|||z|zz|z}tj|}||||fS)zMinimize a 1-D quadratic function subject to bounds. The free term `c` is 0 by default. Bounds must be finite. Returns ------- t : float Minimum point. y : float Minimum value. rg)appendr asarrayrJ) rrlbubrrQextremumy min_indexs rminimize_quadratic_1drl.s RAAvv!8a<     2      HHX    1 A QUQY!A ! I Y<9 %%rc|jdkrH||}tj||}||tj||z|z }nT||j}tj|dzd}||tj||dzzdz }tj||}d|z|zS)aCompute values of a quadratic function arising in least squares. The function is 0.5 * s.T * (J.T * J + diag) * s + g.T * s. Parameters ---------- J : ndarray, sparse matrix or LinearOperator, shape (m, n) Jacobian matrix, affects the quadratic term. g : ndarray, shape (n,) Gradient, defines the linear term. s : ndarray, shape (k, n) or (n,) Array containing steps as rows. diag : ndarray, shape (n,), optional Addition diagonal part, affects the quadratic term. If None, assumed to be 0. Returns ------- values : ndarray with shape (k,) or float Values of the function. If `s` was 2-D, then ndarray is returned, otherwise, float is returned. r-Nr rrCr,)ndimrr Tr#)r^rLrr_Jsrls revaluate_quadraticrrEs. v{{ UU1XX F2rNN   D!$$ $A UU13ZZ F2q5q ! ! !   q!t !,,, ,A q! A 7Q;rc@tj||k||kzS)z$Check if a point lies within bounds.)r all)rrgrhs r in_boundsruos 617qBw' ( ((rc tj|}||}tj|}|tjtjd5tj||z ||z ||z ||z ||<dddn #1swxYwYtj|}|tj||tj | tzfS)aCompute a min_step size required to reach a bound. The function computes a positive scalar t, such that x + s * t is on the bound. Returns ------- step : float Computed step. Non-negative value. hits : ndarray of int with shape of x Each element indicates whether a corresponding variable reaches the bound: * 0 - the bound was not hit. * -1 - the lower bound was hit. * 1 - the upper bound was hit. ignore)overN) r nonzero empty_likefillinferrstatemaximumminequalsignastypeint)rrrgrhnon_zero s_non_zerostepsmin_steps rstep_size_to_boundrts/$z!}}H8J M!  E JJrv ( # # #FF*b1fh%7*%D&(1fh%7*%DFFhFFFFFFFFFFFFFFFve}}H RXeX..1B1B31G1GG GGs%1B""B&)B&绽|=ctj|t}|dkrd|||k<d|||k<|S||z }||z }|tjdtj|z}|tjdtj|z}tj||tj||kz} d|| <tj||tj||kz} d|| <|S)aDetermine which constraints are active in a given point. The threshold is computed using `rtol` and the absolute value of the closest bound. Returns ------- active : ndarray of int with shape of x Each component shows whether the corresponding constraint is active: * 0 - a constraint is not active. * -1 - a lower bound is active. * 1 - a upper bound is active. dtyperr+r-)r zeros_likerr~r1isfiniteminimum) rrgrhr7active lower_dist upper_distlower_thresholdupper_threshold lower_active upper_actives rfind_active_constraintsrs]1C ( ( (F qyyqBwqBw RJaJRZ26"::666ORZ26"::666OKOO2:j/#J#JJLLF<KOO2:j/#J#JJLLF< Mrc ||}t||||}tj|d}tj|d}|dkrItj||||||<tj||||||<nx|||tjdtj||zz||<|||tjdtj||zz ||<||k||kz}d||||zz||<|S)zShift a point to the interior of a feasible region. Each element of the returned vector is at least at a relative distance `rstep` from the closest bound. If ``rstep=0`` then `np.nextafter` is used. r+r-rr,)copyrr r nextafterr~r1) rrgrhrstepx_newr lower_mask upper_mask tight_boundss rmake_strictly_feasiblers1 FFHHE $QB 6 6F&"%%J&!$$J zzLJJHHjLJJHHj ^"RZ26"Z.3I3I%J%JJKj ^"RZ26"Z.3I3I%J%JJKjBJ52:.LL!1B|4D!DEE, Lrc*tj|}tj|}|dktj|z}||||z ||<d||<|dktj|z}||||z ||<d||<||fS)a4Compute Coleman-Li scaling vector and its derivatives. Components of a vector v are defined as follows:: | ub[i] - x[i], if g[i] < 0 and ub[i] < np.inf v[i] = | x[i] - lb[i], if g[i] > 0 and lb[i] > -np.inf | 1, otherwise According to this definition v[i] >= 0 for all i. It differs from the definition in paper [1]_ (eq. (2.2)), where the absolute value of v is used. Both definitions are equivalent down the line. Derivatives of v with respect to x take value 1, -1 or 0 depending on a case. Returns ------- v : ndarray with shape of x Scaling vector. dv : ndarray with shape of x Derivatives of v[i] with respect to x[i], diagonal elements of v's Jacobian. References ---------- .. [1] M.A. Branch, T.F. Coleman, and Y. Li, "A Subspace, Interior, and Conjugate Gradient Method for Large-Scale Bound-Constrained Minimization Problems," SIAM Journal on Scientific Computing, Vol. 21, Number 1, pp 1-23, 1999. rr+r-)r ones_likerr)rrLrgrhradvmasks rCL_scaling_vectorrs< QA q  B ER[__ $Dh4 AdGBtH ER[__ $Dg4 AdGBtH b5LrcJt|||r|tj|fStj|}tj|}|}tj|t }||z}tj||d||z||z ||<||||k||<||z}tj||d||z||z ||<||||k||<||z}||z }tj ||||z d||z} ||tj| d||z| z z||<| ||k||<tj|} d| |<|| fS)z3Compute reflective transformation and its gradient.rr r+) rur rrrrboolr~r remainder) rjrgrh lb_finite ub_finiter g_negativerrrQrLs rreflective_transformationrsB"",q//!! BI BI Aq---J  z !Dj4!bh,4"899AdGwD)Jt : !Dj4!bh,4"899AdGwD)Jt y D RA QtWr$x'QtW55AhAq1T7{Q777AdG1T7{Jt QAAjM a4Krc TtddddddddS)Nz${:^15}{:^15}{:^15}{:^15}{:^15}{:^15} Iterationz Total nfevCostCost reduction Step norm OptimalityprintformatrWrrprint_header_nonlinearr!s> 0 6+|V5E| - -.....rc h|d}n|d}|d}n|d}t|d|d|d|||ddSNz z^15.2ez^15z^15.4er) iterationnfevcostcost_reductionrZ optimalitys rprint_iteration_nonlinearr'sq!*33  ))  Y a aD a ad a a> a9 aj a a abbbbbrc RtdddddddS)Nz{:^15}{:^15}{:^15}{:^15}{:^15}rrrrrrrWrrprint_header_linearr6s< * 6+v'7  !!!!!rcb|d}n|d}|d}n|d}t|d|d|||ddSrr)rrrrZrs rprint_iteration_linearr<si!*33  ))  Y W WD W W W WJ W W WXXXXXrct|tr||S|j|S)z4Compute gradient of the least-squares cost function.) isinstancer rmatvecror)r^rOs r compute_gradrNs6!^$$yy||swwqzzrcJt|rQtj|dddz}ntj|dzddz}| d||dk<ntj||}d|z |fS)z5Compute variables scale based on the Jacobian matrix.r rrCr,Nr-)rr rfpowerr#ravelr~)r^ scale_inv_old scale_invs rcompute_jac_scalerVs{{.Jqwwqzz~~1~5566<<>>C F1a4a(((#- $% )q.!!Jy-88 y=) ##rcxtfd}fd}fd}tj|||S)z#Return diag(d) J as LinearOperator.c4|zSN)matvecrr^rs rrz(left_multiplied_operator..matvecis188A;;rc\ddtjf|zSr)r newaxismatmatXr^rs rrz(left_multiplied_operator..matmatls'BJ!((1++--rcX|zSr)rrrs rrz)left_multiplied_operator..rmatvecos!yyQ'''rrrrr r shaper^rrrrs`` rleft_multiplied_operatorresA......(((((( !'&") + + ++rcxtfd}fd}fd}tj|||S)z#Return J diag(d) as LinearOperator.cXtj|zSr)rr rrs rrz)right_multiplied_operator..matveczs!xx a(((rc\|ddtjfzSr)rr rrs rrz)right_multiplied_operator..matmat}s)xxAaaam,,---rc4|zSrrrs rrz*right_multiplied_operator..rmatvecs199Q<<rrrrs`` rright_multiplied_operatorrvsA))))))......       !'&") + + ++rctj\}fd}fd}t|z|f||S)zReturn a matrix arising in regularized least squares as LinearOperator. The matrix is [ J ] [ D ] where D is diagonal matrix with elements from `diag`. c\tj||zfSr)r hstackr)rr^r_s rrz(regularized_lsq_operator..matvecs&y!((1++tax0111rcb|d}|d}||zzSrr)rx1x2r^r_r3s rrz)regularized_lsq_operator..rmatvecs6 rrU qrrUyy}}tby((r)rr)r rr )r^r_r2rrr3s`` @rregularized_lsq_operatorrs A 7DAq222222))))))) 1q5!*VW E E EErTc&|r)t|ts|}t|r+|xj||jdzc_n+t|trt||}n||z}|S)zhCompute J diag(d). If `copy` is False, `J` is modified in place (unless being LinearOperator). clip)mode)rr rrdatatakeindicesrr^rrs rright_multiplyrs  Jq.11 FFHH{{ !&&&000 A~ & & %a + + Q Hrcn|r)t|ts|}t|r;|xjt j|t j|jzc_n?t|trt||}n||ddt j fz}|S)zhCompute diag(d) J. If `copy` is False, `J` is modified in place (unless being LinearOperator). N) rr rrrr repeatdiffindptrrrrs r left_multiplyrs  Jq.11 FFHH{{ ")Arwqx00111 A~ & & $Q * * Qqqq"*}  HrcX|||zko|dk}||||zzk}|r|rdS|rdS|rdSdS)z8Check termination condition for nonlinear least squares.rVr r"NrW) dFFdx_normx_normr?ftolxtolftol_satisfiedxtol_satisfieds rcheck_terminationrs^$(]3ut|Nttf}55N.q q qtrc|dd|dz|dzzz}t||tk<|dz}||d|z z}t||d|fS)z`Scale Jacobian and residuals for a robust loss function. Arrays are modified in place. r-r r,F)r)r.r)r^rOrhoJ_scales rscale_for_robust_loss_functionrsg !fq3q6zAqD((G GGcM OGQ' A G% 0 0 0! 33r)Nrr)NN)rr)r)T).__doc__mathrnumpyr numpy.linalgr scipy.linalgrrr scipy.sparserscipy.sparse.linalgr r finfofloatepsr.rr@rTr\rcrlrrrurrrrrrrrrrrrrrrrrrrWrrrsv11;;;;;;;;;;!!!!!!@@@@@@@@bhuoo $$$NAE/1ooood000f:0000f&&&&.$$$$T))) HHH:$$$$N6)))XD... c c c!!! Y Y Y$ $ $ $ $+++"+++"FFF,    $    $    4 4 4 4 4r