^MhOjdZddlmZddlmZddlZgdZGddeZ d Z d dZ d!dZ e Z d"dZ d#dZdZdZdZd$dZd$dZd%dZ d&dZ d'dZdS)(z Functions --------- .. autosummary:: :toctree: generated/ line_search_armijo line_search_wolfe1 line_search_wolfe2 scalar_search_wolfe1 scalar_search_wolfe2 )warn)DCSRCHN)LineSearchWarningline_search_wolfe1line_search_wolfe2scalar_search_wolfe1scalar_search_wolfe2line_search_armijoceZdZdS)rN)__name__ __module__ __qualname__Z/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/scipy/optimize/_linesearch.pyrrsDrrcPd|cxkr |cxkrdksntddS)Nrrz.'c1' and 'c2' do not satisfy'0 < c1 < c2 < 1'.) ValueError)c1c2s r _check_c1_c2rsA OOOOROOOO!OOOO.// / Orr-C6??2:0yE>+=c | gR}|gdgdgfd} fd}tj|}t| |||||| | | |  \}}}|dd||dfS)a1 As `scalar_search_wolfe1` but do a line search to direction `pk` Parameters ---------- f : callable Function `f(x)` fprime : callable Gradient of `f` xk : array_like Current point pk : array_like Search direction gfk : array_like, optional Gradient of `f` at point `xk` old_fval : float, optional Value of `f` at point `xk` old_old_fval : float, optional Value of `f` at point preceding `xk` The rest of the parameters are the same as for `scalar_search_wolfe1`. Returns ------- stp, f_count, g_count, fval, old_fval As in `line_search_wolfe1` gval : array Gradient of `f` at the final point Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. NrcBdxxdz cc<|zzgRSNrrr)sargsffcpkxks rphizline_search_wolfe1..phiRs8 1 qad"T""""rc~|zzgRd<dxxdz cc<tjdSrnpdot)r r!fprimegcgvalr$r%s rderphiz"line_search_wolfe1..derphiVsQ&ad*T***Q 1 vd1gr"""r)rramaxaminxtol)r)r*r )r"r+r%r$gfkold_fval old_old_fvalr!rrr/r0r1r&r.derphi0stpfvalr#r,r-s```` ` @@@rrr%sL {fR$ 5D B B################### fS"ooG. <bt$T;;;Cx 1r!udHd1g 55rc t||| |d}| |d}|(|dkr"tdd||z z|z } | dkrd} nd} d} t||||| ||} | | ||| \} }}}| ||fS)a Scalar function search for alpha that satisfies strong Wolfe conditions alpha > 0 is assumed to be a descent direction. Parameters ---------- phi : callable phi(alpha) Function at point `alpha` derphi : callable phi'(alpha) Objective function derivative. Returns a scalar. phi0 : float, optional Value of phi at 0 old_phi0 : float, optional Value of phi at previous point derphi0 : float, optional Value derphi at 0 c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax, amin : float, optional Maximum and minimum step size xtol : float, optional Relative tolerance for an acceptable step. Returns ------- alpha : float Step size, or None if no suitable step was found phi : float Value of `phi` at the new point `alpha` phi0 : float Value of `phi` at `alpha=0` Notes ----- Uses routine DCSRCH from MINPACK. Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1`` as described in [1]_. References ---------- .. [1] Nocedal, J., & Wright, S. J. (2006). Numerical optimization. In Springer Series in Operations Research and Financial Engineering. (Springer Series in Operations Research and Financial Engineering). Springer Nature. Nr?)\(@d)phi0r5maxiter)rminr)r&r.r=old_phi0r5rrr/r0r1alpha1r>dcsrchr6phi1tasks rr r dsjR |s2ww&**1 S&$/27:;; A::FG CRtT : :F"FT7GCtT d?r c b dgdgdgdgfd} |fd| gR}tj|}  fd}nd}t| ||||| | ||  \}}}}|tdtd nd}|dd|||fS) a Find alpha that satisfies strong Wolfe conditions. Parameters ---------- f : callable f(x,*args) Objective function. myfprime : callable f'(x,*args) Objective function gradient. xk : ndarray Starting point. pk : ndarray Search direction. The search direction must be a descent direction for the algorithm to converge. gfk : ndarray, optional Gradient value for x=xk (xk being the current parameter estimate). Will be recomputed if omitted. old_fval : float, optional Function value for x=xk. Will be recomputed if omitted. old_old_fval : float, optional Function value for the point preceding x=xk. args : tuple, optional Additional arguments passed to objective function. c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax : float, optional Maximum step size extra_condition : callable, optional A callable of the form ``extra_condition(alpha, x, f, g)`` returning a boolean. Arguments are the proposed step ``alpha`` and the corresponding ``x``, ``f`` and ``g`` values. The line search accepts the value of ``alpha`` only if this callable returns ``True``. If the callable returns ``False`` for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions. maxiter : int, optional Maximum number of iterations to perform. Returns ------- alpha : float or None Alpha for which ``x_new = x0 + alpha * pk``, or None if the line search algorithm did not converge. fc : int Number of function evaluations made. gc : int Number of gradient evaluations made. new_fval : float or None New function value ``f(x_new)=f(x0+alpha*pk)``, or None if the line search algorithm did not converge. old_fval : float Old function value ``f(x0)``. new_slope : float or None The local slope along the search direction at the new value ````, or None if the line search algorithm did not converge. Notes ----- Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, 'Numerical Optimization', 1999, pp. 59-61. The search direction `pk` must be a descent direction (e.g. ``-myfprime(xk)``) to find a step length that satisfies the strong Wolfe conditions. If the search direction is not a descent direction (e.g. ``myfprime(xk)``), then `alpha`, `new_fval`, and `new_slope` will be None. Examples -------- >>> import numpy as np >>> from scipy.optimize import line_search A objective function and its gradient are defined. >>> def obj_func(x): ... return (x[0])**2+(x[1])**2 >>> def obj_grad(x): ... return [2*x[0], 2*x[1]] We can find alpha that satisfies strong Wolfe conditions. >>> start_point = np.array([1.8, 1.7]) >>> search_gradient = np.array([-1.0, -1.0]) >>> line_search(obj_func, obj_grad, start_point, search_gradient) (1.0, 2, 1, 1.1300000000000001, 6.13, [1.6, 1.4]) rNcBdxxdz cc<|zzgRSrr)alphar!r"r#r$r%s rr&zline_search_wolfe2..phis8 1 qebj(4((((rcdxxdz cc<|zzgRd<|d<tjdSrr()rHr!r+r,r- gval_alphar$r%s rr.z"line_search_wolfe2..derphi#sZ 1 &ebj04000Q 1 vd1gr"""rcjd|kr ||zz}|||dS)Nrr) rHr&xr.extra_conditionr-rJr$r%s rextra_condition2z,line_search_wolfe2..extra_condition20sF!}%%u URZA"?5!S$q':: :r)r>*The line search algorithm did not converge stacklevel)r)r*r rr)r"myfprimer%r$r2r3r4r!rrr/rMr>r&r5rN alpha_starphi_star derphi_starr.r#r+r,r-rJs` `` ` ` @@@@@@rrrs| B B 6DJ)))))))))F###########  {fR$fS"ooG" ; ; ; ; ; ; ; ; ; ; ;  2F <"b$ g3/3/3//J(K 9 1 . . . . .1g r!ubeXx DDrc rt||| |d}| |d}d} | |dkrtdd||z z|z } nd} | dkrd} |t| |} || } |} |}|d}t| D]}| dks|8| |kr2d}|}|}d}| dkrd}ndd |z}t|td n|dk}| ||| z|zzks| | kr"|r t | | | | |||||||| \}}}n|| }t || |zkr|| | r| }| }|}no|dkr t | | | | |||||||| \}}}nId | z}|t||}| } |} | } || } |}| }| }d}td td ||||fS) aFind alpha that satisfies strong Wolfe conditions. alpha > 0 is assumed to be a descent direction. Parameters ---------- phi : callable phi(alpha) Objective scalar function. derphi : callable phi'(alpha) Objective function derivative. Returns a scalar. phi0 : float, optional Value of phi at 0. old_phi0 : float, optional Value of phi at previous point. derphi0 : float, optional Value of derphi at 0 c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax : float, optional Maximum step size. extra_condition : callable, optional A callable of the form ``extra_condition(alpha, phi_value)`` returning a boolean. The line search accepts the value of ``alpha`` only if this callable returns ``True``. If the callable returns ``False`` for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions. maxiter : int, optional Maximum number of iterations to perform. Returns ------- alpha_star : float or None Best alpha, or None if the line search algorithm did not converge. phi_star : float phi at alpha_star. phi0 : float phi at 0. derphi_star : float or None derphi at alpha_star, or None if the line search algorithm did not converge. Notes ----- Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, 'Numerical Optimization', 1999, pp. 59-61. Nr9rr:r;cdS)NTr)rHr&s rrMz-scalar_search_wolfe2..extra_conditions4rz7Rounding errors prevent the line search from convergingz4The line search algorithm could not find a solution zless than or equal to amax: rPrQrO)rr?rangerr_zoomabs)r&r.r=r@r5rrr/rMr>alpha0rAphi_a1phi_a0 derphi_a0irTrUrVmsgnot_first_iteration derphi_a1alpha2s rr r IspR |s2ww&** F1 S&$/27:;; zz VT"" S[[FFI   7^^9.9. Q;;4+ JHDK{{OL;T;;< 'A 6 6 6 6 E!e TBK'11 1 1 v  #6 fff$if"GR_FF .J+ EF6NN  NNrc'k ) )vv.. # !'  NNfff$if"GR_FF .J+ EV  &&FV    9 1 . . . . x{ 22rc tjddd5 |}||z }||z } || zdz|| z z} tjd} | dz| d<|dz | d<| dz | d<|dz| d <tj| tj||z ||zz ||z || zz g\} } | | z} | | z} | | zd| z|zz }|| tj|zd| zz z}n#t$rYd d d d SwxYw d d d n #1swxYwYtj|sd S|S) z Finds the minimizer for a cubic polynomial that goes through the points (a,fa), (b,fb), and (c,fc) with derivative at a of fpa. If no minimizer can be found, return None. raisedivideoverinvalidrP)rPrP)rr)rr)rr)rrN) r)errstateemptyr*asarrayflattensqrtArithmeticErrorisfinite)afafpabfbcr#Cdbdcdenomd1ABradicalxmins r _cubicminrs G'7 C C C AQBQB"WNb2g.E&!!BQwBtHaxBtHaxBtHQwBtHVB BGa"f,<,.Ga"f,<,>!?!??FwyyJJFQ JA JA!ea!eai'GRWW---!a%88DD   %"  !& ;t  t Ks5DCC43D4 D >D D  DD!$D!ctjddd5 |}|}||dzz }||z ||zz ||zz }||d|zz z } n#t$rYddddSwxYw dddn #1swxYwYtj| sdS| S)z Finds the minimizer for a quadratic polynomial that goes through the points (a,fa), (b,fb) with derivative at a of fpa. rfrgr:@N)r)rlrqrr) rsrtrurvrwDryrzrrs r_quadminrs# G'7 C C C AAQWBa!b&R"W-AqC!G}$DD       ;t  t Ks4A,(AA, A A,AA,,A03A0c Ld} d} d}d}|}d} ||z }|dkr||}}n||}}| dkr||z}t|||||||}| dks||||z ks |||zkr4||z}t|||||}||||z ks |||zkr|d|zz}||}||| |z|zzks||kr |}|}|}|}nT||}t|| |zkr| ||r|}|}|}n3|||z zdkr |}|}|}|}n|}|}|}|}|}| dz } | | krd}d}d}n|||fS) a Zoom stage of approximate linesearch satisfying strong Wolfe conditions. Part of the optimization algorithm in `scalar_search_wolfe2`. Notes ----- Implements Algorithm 3.6 (zoom) in Wright and Nocedal, 'Numerical Optimization', 1999, pp. 61. rErg?皙?TN?r)rrr[)a_loa_hiphi_lophi_hi derphi_lor&r.r=r5rrrMr>r`delta1delta2phi_reca_recdalpharsrvcchka_jqchkphi_aj derphi_aja_starval_star valprime_stars rrZrZsG A F FG E@ A::qAAqA EEF?DD&)T6!7,,C FF q4xS1t8^^F?D4D&AAC qv34<<SZ'S TBsF7N* * *&0@0@GEDFFs I9~~"W,,f1M1M,! ) $+&!++  DF!I Q KKFH M A@B 8] **rc tjdg  fd}| |d} n|} tj|} t|| | ||\} } | d| fS)aMinimize over alpha, the function ``f(xk+alpha pk)``. Parameters ---------- f : callable Function to be minimized. xk : array_like Current point. pk : array_like Search direction. gfk : array_like Gradient of `f` at point `xk`. old_fval : float Value of `f` at point `xk`. args : tuple, optional Optional arguments. c1 : float, optional Value to control stopping criterion. alpha0 : scalar, optional Value of `alpha` at start of the optimization. Returns ------- alpha f_count f_val_at_alpha Notes ----- Uses the interpolation algorithm (Armijo backtracking) as suggested by Wright and Nocedal in 'Numerical Optimization', 1999, pp. 56-57 rcBdxxdz cc<|zzgRSrr)rAr!r"r#r$r%s rr&zline_search_armijo..phis8 1 qfRi'$''''rNr9)rr\)r) atleast_1dr*scalar_search_armijo)r"r%r$r2r3r!rr\r&r=r5rHrCr#s``` ` @rr r osD r  B B(((((((((s2wwfS"ooG&sD'b.4666KE4 "Q% rc `t||||||||}|d|dd|dfS)z8 Compatibility wrapper for `line_search_armijo` )r!rr\rrrP)r ) r"r%r$r2r3r!rr\rs rline_search_BFGSrsD 1b"c8$2"( * * *A Q41q!A$ rc||}||||z|zzkr||fS| |dzzdz ||z ||zz z }||}||||z|zzkr||fS||kr|dz|dzz||z z} |dz||z ||zz z|dz||z ||zz zz } | | z } |dz ||z ||zz z|dz||z ||zz zz} | | z } | tjt| dzd| z|zz zd| zz } || } | ||| z|zzkr| | fS|| z |dz ks d| |z z dkr|dz } |}| }|}| }||kd|fS)a(Minimize over alpha, the function ``phi(alpha)``. Uses the interpolation algorithm (Armijo backtracking) as suggested by Wright and Nocedal in 'Numerical Optimization', 1999, pp. 56-57 alpha > 0 is assumed to be a descent direction. Returns ------- alpha phi1 rPrrkg@rgQ?N)r)rpr[)r&r=r5rr\r0r^rAr]factorrsrvrdphi_a2s rrrsS[[F 6 '))))v~Z&!) #c )Vd]Wv=M-M NF S[[F$F7****v~ 4--VQY&&-8 AI$7 8 AI$7 8 9 J QYJ&4-'&.8 9 AI$7 8 9 J"rws1a4!a%'/#9::;;;AFV dRYw.. . .6> ! VOv| + +F6M0AT/I/Ic\F+ 4--0 <rrrc|d}t|} d} d} d} || |zz} || \}}|| |z|| dzz|zz kr| } n| dz|z|d| zdz |zzz }|| |zz } || \}}|| |z|| dzz|zz kr| } nP| dz|z|d| zdz |zzz }tj||| z|| z} tj||| z|| z} | | ||fS)a@ Nonmonotone backtracking line search as described in [1]_ Parameters ---------- f : callable Function returning a tuple ``(f, F)`` where ``f`` is the value of a merit function and ``F`` the residual. x_k : ndarray Initial position. d : ndarray Search direction. prev_fs : float List of previous merit function values. Should have ``len(prev_fs) <= M`` where ``M`` is the nonmonotonicity window parameter. eta : float Allowed merit function increase, see [1]_ gamma, tau_min, tau_max : float, optional Search parameters, see [1]_ Returns ------- alpha : float Step length xp : ndarray Next position fp : float Merit function value at next position Fp : ndarray Residual at next position References ---------- [1] "Spectral residual method without gradient information for solving large-scale nonlinear systems of equations." W. La Cruz, J.M. Martinez, M. Raydan. Math. Comp. **75**, 1429 (2006). rTrP)maxr)clip)r"x_kdprev_fsetagammatau_mintau_maxf_kf_baralpha_palpha_mrHxpfpFpalpha_tpalpha_tms r_nonmonotone_line_search_cruzrs]P "+C LLEGG EJ 7Q; 2B uwz1C77 7 7E A:#rQwY]C,?'?@ 7Q; 2B uwz1C77 7 7HE A:#rQwY]C,?'?@'(Gg$5w7HII'(Gg$5w7HII)J, "b" r333333?c d} d} d} || |zz}||\}}|||z|| dzz|zz kr| } n| dz|z|d| zdz |zzz }|| |zz }||\}}|||z|| dzz|zz kr| } nP| dz|z|d| zdz |zzz }tj||| z| | z} tj||| z| | z} | |zdz}| |z||zz|z|z }|}| |||||fS)a Nonmonotone line search from [1] Parameters ---------- f : callable Function returning a tuple ``(f, F)`` where ``f`` is the value of a merit function and ``F`` the residual. x_k : ndarray Initial position. d : ndarray Search direction. f_k : float Initial merit function value. C, Q : float Control parameters. On the first iteration, give values Q=1.0, C=f_k eta : float Allowed merit function increase, see [1]_ nu, gamma, tau_min, tau_max : float, optional Search parameters, see [1]_ Returns ------- alpha : float Step length xp : ndarray Next position fp : float Merit function value at next position Fp : ndarray Residual at next position C : float New value for the control parameter C Q : float New value for the control parameter Q References ---------- .. [1] W. Cheng & D.-H. Li, ''A derivative-free nonmonotone line search and its application to the spectral residual method'', IMA J. Numer. Anal. 29, 814 (2009). rTrP)r)r)r"rrrryQrrrrnurrrHrrrrrQ_nexts r_nonmonotone_line_search_chengr2s{^GG EJ 7Q; 2B S57A:-33 3 3E A:#rQwY]C,?'?@ 7Q; 2B S57A:-33 3 3HE A:#rQwY]C,?'?@'(Gg$5w7HII'(Gg$5w7HII)J.!VaZF a1s7 b F*AA "b"a ""r) NNNrrrrrr)NNNrrrrr) NNNrrrNNrE)NNNrrNNrE)rrr)rrr)rrr)rrrr)__doc__warningsr_dcsrchrnumpyr)__all__RuntimeWarningrrrr line_searchrr rrrZr rrrrrrrrs   ! ! !        ////337?C!<6<6<6<6~IM%(27JJJJZ! @DIM57LELELELE^,004/379Q3Q3Q3Q3hD*T+T+T+v1111h7777~DGEEEEREH&*N#N#N#N#N#N#r