^Mh@gdZdZddlZddlZddlZddlZddlmZmZm Z m Z m Z m Z ddlZ ddlmZmZmZddlmZddlmZmZmZmZdd lmZdd lmZdd lmZmZm Z m!Z!m"Z"dd l#m$Z$m%Z%dd l&m'Z'ddl(m)Z*dddddddZ+GddZ,dfdZ-Gdde Z.Gdde/Z0dZ1dZ2dZ3e e j4e5j6Z7dgd!Z8 dhd#Z9d$Z:d%Z;d&Zd)Z?d*Z@Gd+d,eAZBd-ZC did/ZD djd1ZEe7fd2ZFe"d3d45e7d6dd7d8ZGd9ZHGd:d;eAZId<ZJdd"d=e jKe7ddddddd.d>dfd?ZLd"ddd=e jKe7dd0d0ddd.d>dfd@ZMdfdAZNdd"d=e jKe7dddddd.dBf dCZOd"ddd=e jKe7dd0d0dd.dBf dDZPddd"d=e7dddddd.d>f dEZQd"ddddd=e7dd0d0d.d>f dFZR dkdHZS dldIZTGdJdKZUdmdMZV dndNZWd"de7ddOfdPZXdd"e7dOdfdQZYdodVZZGdWdXeAZ[dYZ\dZZ] dpd\Z^ did]Z_ dqd^Z`d_Zad"d`deDd0dfdaZbGdbdcZcdrdeZddS)s)fmin fmin_powell fmin_bfgsfmin_ncgfmin_cg fminboundbrentgoldenbracketrosen rosen_der rosen_hessrosen_hess_prodbrute approx_fprime line_search check_gradOptimizeResult show_optionsOptimizeWarningzrestructuredtext enN)eyeargminzerosshapeasarraysqrt)cholesky issymmetric LinAlgError)LinearOperator)line_search_wolfe1line_search_wolfe2r#LineSearchWarning)approx_derivative)getfullargspec_no_self) MapWrappercheck_random_state _RichResult_call_callback_maybe_halt_transition_to_rng)ScalarFunction FD_METHODS)array_namespace)array_api_extraz%Optimization terminated successfully.z9Maximum number of function evaluations has been exceeded.z/Maximum number of iterations has been exceeded.z=Desired error not necessarily achieved due to precision loss.zNaN result encountered.z-The result is outside of the provided bounds.)successmaxfevmaxiterpr_lossnan out_of_boundsc*eZdZdZdZdZdZdZdS) MemoizeJaczkDecorator that caches the return values of a function returning ``(fun, grad)`` each time it is called.c>||_d|_d|_d|_dSN)funjac_valuex)selfr:s X/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/scipy/optimize/_optimize.py__init__zMemoizeJac.__init__As" ctj||jkr|j|jUtj||_|j|g|R}|d|_|d|_dSdS)Nr!r)npallr=r<r;rcopyr:)r>r=argsfgs r?_compute_if_neededzMemoizeJac._compute_if_neededGsva46k"" dk&9TX=MZ]]''))DF!#d###B!uDHQ%DKKK >N=MrAc,|j|g|R|jS)z returns the function value )rHr<r>r=rFs r?__call__zMemoizeJac.__call__Ns%)D)))){rAc,|j|g|R|jSr9)rHr;rJs r? derivativezMemoizeJac.derivativeSs$)D))))xrAN)__name__ __module__ __qualname____doc__r@rHrKrMrAr?r7r7=sZ     rAr7c|dvrStj}t|jdhkrfd}n|dkrfd}n|dkrfd}nfd }d |_|S) zAWrap a user-provided callback so that attributes can be attached.N>tncslsqpcobylacobyqaintermediate_resultc|S)N)rXrRrescallbacks r?wrapped_callbackz(_wrap_callback..wrapped_callback`s8444 4rA trust-constrcJtj|j|Sr9rCrEr=rZs r?r]z(_wrap_callback..wrapped_callbackcs8BGCENNC00 0rAdifferential_evolutioncTtj|j|jSr9)rCrEr= convergencerZs r?r]z(_wrap_callback..wrapped_callbackfs!8BGCENNCO<< .wrapped_callbackis8BGCENN++ +rAF)inspect signatureset parametersstop_iteration)r\methodsigr]s` r?_wrap_callbackrlXs6%III  H % %C 3>4555 5 5 5 5 5 5 > ! ! 1 1 1 1 1 1 + + + = = = = = = , , , , ,',# rAceZdZdZdS)ra Represents the optimization result. Attributes ---------- x : ndarray The solution of the optimization. success : bool Whether or not the optimizer exited successfully. status : int Termination status of the optimizer. Its value depends on the underlying solver. Refer to `message` for details. message : str Description of the cause of the termination. fun : float Value of objective function at `x`. jac, hess : ndarray Values of objective function's Jacobian and its Hessian at `x` (if available). The Hessian may be an approximation, see the documentation of the function in question. hess_inv : object Inverse of the objective function's Hessian; may be an approximation. Not available for all solvers. The type of this attribute may be either np.ndarray or scipy.sparse.linalg.LinearOperator. nfev, njev, nhev : int Number of evaluations of the objective functions and of its Jacobian and Hessian. nit : int Number of iterations performed by the optimizer. maxcv : float The maximum constraint violation. Notes ----- Depending on the specific solver being used, `OptimizeResult` may not have all attributes listed here, and they may have additional attributes not listed here. Since this class is essentially a subclass of dict with attribute accessors, one can see which attributes are available using the `OptimizeResult.keys` method. N)rNrOrPrQrRrAr?rrps((R DrArceZdZdS)rNrNrOrPrRrAr?rrDrArcHd}|||stddSdS)Ncjt|r# t|dS#t$rYdSwxYwdS)NTF)rrr)As r? is_pos_defz,_check_positive_definite..is_pos_defsN q>>   t   uu 5s " 00z+'hess_inv0' matrix isn't positive definite.) ValueError)Hkrts r?_check_positive_definiterwsK ~z"~~ LJKK K~ L LrAc|r[dtt|}t jd|t ddSdS)Nz, zUnknown solver options:  stacklevel)joinmapstrkeyswarningswarnr)unknown_optionsmsgs r?_check_unknown_optionsrsgWiiC!5!5!7!78899  666TUVVVVVV WWrAcZtj|dkotj|S)zJTest whether `x` is either a finite scalar or a finite array scalar. r!)rCsizeisfinite)r=s r?is_finite_scalarrs# 71::? -r{1~~-rAc>|tjkr&tjtj|S|tj kr&tjtj|Stjtj||zdd|z zS)Nraxis?)rCinfamaxabsaminsum)r=ords r?vecnormrst bf}}wrvayy!!! wrvayy!!!vbfQiin1---c ::rArRc t|r|}n|tvrd}|}nd}|}|d}|tj tjf}t ||||||||} | S)a Creates a ScalarFunction object for use with scalar minimizers (BFGS/LBFGSB/SLSQP/TNC/CG/etc). Parameters ---------- fun : callable The objective function to be minimized. ``fun(x, *args) -> float`` where ``x`` is an 1-D array with shape (n,) and ``args`` is a tuple of the fixed parameters needed to completely specify the function. x0 : ndarray, shape (n,) Initial guess. Array of real elements of size (n,), where 'n' is the number of independent variables. jac : {callable, '2-point', '3-point', 'cs', None}, optional Method for computing the gradient vector. If it is a callable, it should be a function that returns the gradient vector: ``jac(x, *args) -> array_like, shape (n,)`` If one of `{'2-point', '3-point', 'cs'}` is selected then the gradient is calculated with a relative step for finite differences. If `None`, then two-point finite differences with an absolute step is used. args : tuple, optional Extra arguments passed to the objective function and its derivatives (`fun`, `jac` functions). bounds : sequence, optional Bounds on variables. 'new-style' bounds are required. eps : float or ndarray If ``jac is None`` the absolute step size used for numerical approximation of the jacobian via forward differences. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x0) * max(1, abs(x0))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. hess : {callable, '2-point', '3-point', 'cs', None} Computes the Hessian matrix. If it is callable, it should return the Hessian matrix: ``hess(x, *args) -> {LinearOperator, spmatrix, array}, (n, n)`` Alternatively, the keywords {'2-point', '3-point', 'cs'} select a finite difference scheme for numerical estimation. Whenever the gradient is estimated via finite-differences, the Hessian cannot be estimated with options {'2-point', '3-point', 'cs'} and needs to be estimated using one of the quasi-Newton strategies. Returns ------- sf : ScalarFunction N2-pointcdSr9rR)r=rFs r?hessz&_prepare_scalar_function..hesss4rA)epsilon)callabler-rCrr,) r:x0r;rFboundsrfinite_diff_rel_steprgradsfs r?_prepare_scalar_functionrsx}}     |    ~6'26" RtT,fg G G GB IrAcfd}|S)Nc:t|}|Sr9) _check_clip_x)r=rfuncs r?evalz_clip_x_for_func..eval/s !V $ $tAwwrArR)rrrs`` r?_clip_x_for_funcr)s)  KrAc||dks||dkr@tjdtdt j||d|d}|S|S)Nrr!zJValues in x were outside bounds during a minimize step, clipping to boundsrz)anyrrRuntimeWarningrCclip)r=rs r?rr6s F1I VAY 3 3 5 5 :$ 4 4 4 4 GAvay&) , , HrActt|}||}||jdr.|||dj}|d|dd|dddzz dzzd|ddz dzzd|j }|S) a The Rosenbrock function. The function computed is:: sum(100.0*(x[1:] - x[:-1]**2.0)**2.0 + (1 - x[:-1])**2.0) Parameters ---------- x : array_like 1-D array of points at which the Rosenbrock function is to be computed. Returns ------- f : float The value of the Rosenbrock function. See Also -------- rosen_der, rosen_hess, rosen_hess_prod Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen >>> X = 0.1 * np.arange(10) >>> rosen(X) 76.56 For higher-dimensional input ``rosen`` broadcasts. In the following example, we use this to plot a 2D landscape. Note that ``rosen_hess`` does not broadcast in this manner. >>> import matplotlib.pyplot as plt >>> from mpl_toolkits.mplot3d import Axes3D >>> x = np.linspace(-1, 1, 50) >>> X, Y = np.meshgrid(x, x) >>> ax = plt.subplot(111, projection='3d') >>> ax.plot_surface(X, Y, rosen([X, Y])) >>> plt.show() integralrgY@r!N@r)rdtype)r.risdtyperastyper)r=xprs r?r r AsT   B 1 A zz!':&&/ IIaB- . . u!""#2# +c11Q3B3Z#4EEQW  & &A HrAc.t|}||}||jdr.|||dj}|dd}|dd}|dd}||}d||dzz zd ||dzz z|zz dd|z zz |dd<d |d z|d|d dzz zdd|d z zz |d <d|d|ddzz z|d<|S) aP The derivative (i.e. gradient) of the Rosenbrock function. Parameters ---------- x : array_like 1-D array of points at which the derivative is to be computed. Returns ------- rosen_der : (N,) ndarray The gradient of the Rosenbrock function at `x`. See Also -------- rosen, rosen_hess, rosen_hess_prod Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen_der >>> X = 0.1 * np.arange(9) >>> rosen_der(X) array([ -2. , 10.6, 15.6, 13.4, 6.4, -3. , -12.4, -19.4, 62. ]) rrr!rNrpr)r.rrrr zeros_like)r=rxmxm_m1xm_p1ders r?r r ts/6   B 1 A zz!':&&/ IIaB- . . 1R4B crcFE abbEE --  CUAX &A &+,./1r6l;C"I AaD[AaD1Q47N +a1qt8n >> import numpy as np >>> from scipy.optimize import rosen_hess >>> X = 0.1 * np.arange(4) >>> rosen_hess(X) array([[-38., 0., 0., 0.], [ 0., 134., -40., 0.], [ 0., -40., 130., -80.], [ 0., 0., -80., 200.]]) r!ndimrrrrNr)offsetrrrrrr)r) r.xpx atleast_ndrrrrcreate_diagonalrr)r=rHdiagonals r?r r sD<   B qqR(((A zz!':&&/ IIaB- . . TAcrcF]1 < < <  sQssV|B2 > > > ?Axx !'x22H1q.31:-1HQKHRL4!AbD'1*,,sQqrrU{:HQrTN s"8333 33rAct||}tj|d|}||jdr.|||dj}|||j}||jd|j}d|ddzzd |dzz dz|dzd |dz|dzz |d<d |d d z|d d zd d|dddzzzd |dd zz |ddzzd |ddz|dd zz |dd<d |d z|d zd|dzz|d<|S)a Product of the Hessian matrix of the Rosenbrock function with a vector. Parameters ---------- x : array_like 1-D array of points at which the Hessian matrix is to be computed. p : array_like 1-D array, the vector to be multiplied by the Hessian matrix. Returns ------- rosen_hess_prod : ndarray The Hessian matrix of the Rosenbrock function at `x` multiplied by the vector `p`. See Also -------- rosen, rosen_der, rosen_hess Examples -------- >>> import numpy as np >>> from scipy.optimize import rosen_hess_prod >>> X = 0.1 * np.arange(9) >>> p = 0.5 * np.arange(9) >>> rosen_hess_prod(X, p) array([ -0., 27., -10., -95., -192., -265., -278., -195., -180.]) r!rrrrrrrrrNrrrr) r.rrrrrrrr)r=prHps r?rrs> A  B qqR(((A zz!':&&/ IIaB- . . 1AG $$A !'!*AG , ,B AaD!G^cAaDj (1 ,! 4sQqTzAaD7H HBqEq"v #2#&ta"gqj((3122;6!AbD'ABa"g !""%&BqtHAbE\AbE !C"I -BrF IrAc0dgdfSfd}|fS)Nrc.dxxdz cc<tj|g|zR}tj|sQ tj|}n)#t t f$r}t d|d}~wwxYw|S)Nrr!@The user-provided objective function must return a scalar value.)rCrEisscalarritem TypeErrorru)r= wrapper_argsfxerFfunctionncallss r?function_wrapperz/_wrap_scalar_function..function_wrappersq Q Xbgajj 9L4$7 9 9 9{2 G GZ^^((**z* G G G "?@@EFG G s&A,,B=B  BrR)rrFrrs`` @r?_wrap_scalar_functionrsOSFt|        # ##rAceZdZdS)_MaxFuncCallErrorNrorRrAr?rrrprArc4dgdfSfd}|fS)Nrcddkrtddxxdz cc<tj|g|zR}tj|sQ tj|}n)#t tf$r}td|d}~wwxYw|S)NrzToo many function callsr!r)rrCrErrrrru)r=rrrrFrmaxfunrs r?rzA_wrap_scalar_function_maxfun_validation..function_wrappers !9  #$=>> >q Q Xbgajj 9L4$7 9 9 9{2 G GZ^^((**z* G G G "?@@EFG G s &BB-B((B-rR)rrFrrrs``` @r?'_wrap_scalar_function_maxfun_validationrsUSFt|" # ##rA-C6?c |||||| | d} t| } t|||fd| i| } |r5| d| d| d| d| df}| r || dfz }|S| r| d| dfS| dS) a( Minimize a function using the downhill simplex algorithm. This algorithm only uses function values, not derivatives or second derivatives. Parameters ---------- func : callable func(x,*args) The objective function to be minimized. x0 : ndarray Initial guess. args : tuple, optional Extra arguments passed to func, i.e., ``f(x,*args)``. xtol : float, optional Absolute error in xopt between iterations that is acceptable for convergence. ftol : number, optional Absolute error in func(xopt) between iterations that is acceptable for convergence. maxiter : int, optional Maximum number of iterations to perform. maxfun : number, optional Maximum number of function evaluations to make. full_output : bool, optional Set to True if fopt and warnflag outputs are desired. disp : bool, optional Set to True to print convergence messages. retall : bool, optional Set to True to return list of solutions at each iteration. callback : callable, optional Called after each iteration, as callback(xk), where xk is the current parameter vector. initial_simplex : array_like of shape (N + 1, N), optional Initial simplex. If given, overrides `x0`. ``initial_simplex[j,:]`` should contain the coordinates of the jth vertex of the ``N+1`` vertices in the simplex, where ``N`` is the dimension. Returns ------- xopt : ndarray Parameter that minimizes function. fopt : float Value of function at minimum: ``fopt = func(xopt)``. iter : int Number of iterations performed. funcalls : int Number of function calls made. warnflag : int 1 : Maximum number of function evaluations made. 2 : Maximum number of iterations reached. allvecs : list Solution at each iteration. See also -------- minimize: Interface to minimization algorithms for multivariate functions. See the 'Nelder-Mead' `method` in particular. Notes ----- Uses a Nelder-Mead simplex algorithm to find the minimum of function of one or more variables. This algorithm has a long history of successful use in applications. But it will usually be slower than an algorithm that uses first or second derivative information. In practice, it can have poor performance in high-dimensional problems and is not robust to minimizing complicated functions. Additionally, there currently is no complete theory describing when the algorithm will successfully converge to the minimum, or how fast it will if it does. Both the ftol and xtol criteria must be met for convergence. Examples -------- >>> def f(x): ... return x**2 >>> from scipy import optimize >>> minimum = optimize.fmin(f, 1) Optimization terminated successfully. Current function value: 0.000000 Iterations: 17 Function evaluations: 34 >>> minimum[0] -8.8817841970012523e-16 References ---------- .. [1] Nelder, J.A. and Mead, R. (1965), "A simplex method for function minimization", The Computer Journal, 7, pp. 308-313 .. [2] Wright, M.H. (1996), "Direct Search Methods: Once Scorned, Now Respectable", in Numerical Analysis 1995, Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis, D.F. Griffiths and G.A. Watson (Eds.), Addison Wesley Longman, Harlow, UK, pp. 191-208. )xatolfatolr2r1disp return_allinitial_simplexr\r=r:nitnfevstatusallvecs)rl_minimize_neldermead)rrrFxtolftolr2r full_outputrretallr\roptsr[retlists r?rr-sN .  0 0Dh''H tR I I ID I IC c(CJE CKXN  * I) )G  s8S^+ +s8OrAFc t| |}|}tj|}tj|jtjr|jn tj}tj||}| r:tt|}d}dd|z z}ddd|zz z }dd|z z }nd}d}d}d}d}d}| | j | j }}||k rtd tj ||kstj ||krtjd t"d | tj|||}|t|}tj|dz|f|j}||d <t)|D]C}tj|d}||d krd|z||z||<n|||<|||dz<Dntj|}tj|jtjr|jn tj}tj||}|jdks|jd |jddzkrtdt||jdkrtd|jd}|r |d g}| | |dz}|dz}nI|#|tjkr|dz}n1tj}n$|"|tjkr|dz}n tj}| 8||k}tj|d|z|z |}tj|||}t9t)d|dz} tj|dzftjt}!t=|||\}"} t)|dzD]}||||!|<n#t>$rYnwxYwtj |!}#tj!||#d }tj!|!|#d }!nE#tj |!}#tj!||#d }tj!|!|#d }!wxYwtj |!}#tj!|!|#d }!tj!||#d }d}$|"d |krR|$|krK tj"tj#tj$|dd|d z | krtj"tj$|!d |!ddz | kr tj |!}#tj!||#d }tj!|!|#d }!|r|%|d tM|d |!d }%tO||%rn5n3tj()|ddd |z }&d|z|&z||dzz }'| tj|'||}'||'}(d })|(|!d kr[d||zz|&z||z|dzz }*| tj|*||}*||*}+|+|(kr |*|d<|+|!d<n-|'|d<|(|!d<n!|(|!dkr |'|d<|(|!d<n |(|!dkrQd||zz|&z||z|dzz },| tj|,||},||,}-|-|(kr |,|d<|-|!d<nSd})nPd|z |&z||dzz}.| tj|.||}.||.}/|/|!dkr |.|d<|/|!d<nd})|)rZ| D]W}0|d |||0|d z zz||0<| tj||0||||0<|||0|!|0<X|$dz }$n#t>$rYnwxYwtj |!}#tj!||#d }tj!|!|#d }!|r|%|d tM|d |!d }%tO||%rnn#tj |!}#tj!||#d }tj!|!|#d }!|r|%|d tM|d |!d }%tO||%rYnwxYw|"d |kr|$|kK|d }1tj*|!}2d }3|"d |kr.d}3tVd}4|rtj|4tXd n|$|kr.d}3tVd}4|rtj|4tXd n[tVd}4|rLt[|4t[d|2dt[d|$zt[d|"d ztM|2|$|"d |3|3d k|4|1||!f}5|r||5d<|5S)aj Minimization of scalar function of one or more variables using the Nelder-Mead algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter, maxfev : int Maximum allowed number of iterations and function evaluations. Will default to ``N*200``, where ``N`` is the number of variables, if neither `maxiter` or `maxfev` is set. If both `maxiter` and `maxfev` are set, minimization will stop at the first reached. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. initial_simplex : array_like of shape (N + 1, N) Initial simplex. If given, overrides `x0`. ``initial_simplex[j,:]`` should contain the coordinates of the jth vertex of the ``N+1`` vertices in the simplex, where ``N`` is the dimension. xatol : float, optional Absolute error in xopt between iterations that is acceptable for convergence. fatol : number, optional Absolute error in func(xopt) between iterations that is acceptable for convergence. adaptive : bool, optional Adapt algorithm parameters to dimensionality of problem. Useful for high-dimensional minimization [1]_. bounds : sequence or `Bounds`, optional Bounds on variables. There are two ways to specify the bounds: 1. Instance of `Bounds` class. 2. Sequence of ``(min, max)`` pairs for each element in `x`. None is used to specify no bound. Note that this just clips all vertices in simplex based on the bounds. References ---------- .. [1] Gao, F. and Han, L. Implementing the Nelder-Mead simplex algorithm with adaptive parameters. 2012. Computational Optimization and Applications. 51:1, pp. 259-277 rr!rg??g?gMb0?NzENelder Mead - one of the lower bounds is greater than an upper bound.0Initial guess is not within the specified boundsrrzrT)rEz5`initial_simplex` should be an array of shape (N+1,N)z5Size of `initial_simplex` is not consistent with `x0`rr=r:rrr1r2r0! Current function value: f Iterations: %d! Function evaluations: %d)r:rrrr0messager= final_simplexr).rrC atleast_1dflatten issubdtyperinexactfloat64rfloatlenlbubrrurrrremptyrangearray atleast_2drErrrwherelistfullrrargsorttakemaxravelrappendrr*addreducemin_status_messagerprint)6rrrFr\r2r1rrrrradaptiverrrrrdimrhochipsisigmanonzdeltzdelt lower_bound upper_boundNsimkyrmskone2np1fsimfcallsind iterationsrXxbarxrfxrdoshrinkxefxexcfxcxccfxccjr=fvalwarnflagrresults6 r?rrs8 l?+++ F F r   " " $ $B bh ;; KBHHE Be $ $ $B CGGnn!C%iQ#YAcE H E #)9fi[ + % * * , , @?@@ @ 6+" # # 9rvb;.>'?'? 9 ML)a 9 9 9 9 WRk 2 2 GGhAqz222Aq  A$'''AtqyyH ad*!!CAJJ  mO,,1133]39bjAAQ rzjE*** 8q==CIaLCIaL1,<<<TUU U r77cil " "TUU U IaL q6(6>c'S  RV  #gGGfGG  bf  WFFVF  KhsAkMC/55gc; 445AE??##G 7AE8RV5 1 1 1D:4vNNLFD%q1u # #Ad3q6llDGG #      jgc3""wtS!$$jgc3""wtS!$$$$$$ *T  C 74a D '#sA  CJ !9v  *w"6"6H rxs122wQ'7 8 899::eCCF26$q'DH"45566%??|*T""C'#sA&&C74a((D 's1v&&&"03q6tAw"G"G"G (3FGG  E6==SbS1--1Dc'T!C#b'M1B!WRk::$r((CHT!W}}#)mt+cCi#b'.AA%[+>>Bd2hh99 CG"DHH CG"DHHb>> CG"DHHT"X~~#)mt3cCi#b'6II!-!#[+!F!FB"d2hh#::&(CG'*DHH'(HH !3w$.s2w>!-"$'#{K"H"HC#tCyy$r(??&)CG'+DHH'(H3!(33A%(Ves1vA.G%GCF%1)+$'FK*F*FA&*d3q6llDGG !OJJ     D *T""C'#sA&&C74a((D 's1v&&&"03q6tAw"G"G"G (3FGG   *T""C'#sA&&C74a((D 's1v&&&"03q6tAw"G"G"G (3FGG S !9v  *w"6"6V AA 6$<d>>> ? ? ? +j8 9 9 9 5q A B B B *6!9#+h!m$'1S$KIIIF$#y Msd)P?Q P  Q P  QAR-B_6G1_65b6 `b`bB d! d!crtj|t}||g|R}t||d|||S)a Finite difference approximation of the derivatives of a scalar or vector-valued function. If a function maps from :math:`R^n` to :math:`R^m`, its derivatives form an m-by-n matrix called the Jacobian, where an element :math:`(i, j)` is a partial derivative of f[i] with respect to ``xk[j]``. Parameters ---------- xk : array_like The coordinate vector at which to determine the gradient of `f`. f : callable Function of which to estimate the derivatives of. Has the signature ``f(xk, *args)`` where `xk` is the argument in the form of a 1-D array and `args` is a tuple of any additional fixed parameters needed to completely specify the function. The argument `xk` passed to this function is an ndarray of shape (n,) (never a scalar even if n=1). It must return a 1-D array_like of shape (m,) or a scalar. Suppose the callable has signature ``f0(x, *my_args, **my_kwargs)``, where ``my_args`` and ``my_kwargs`` are required positional and keyword arguments. Rather than passing ``f0`` as the callable, wrap it to accept only ``x``; e.g., pass ``fun=lambda x: f0(x, *my_args, **my_kwargs)`` as the callable, where ``my_args`` (tuple) and ``my_kwargs`` (dict) have been gathered before invoking this function. .. versionchanged:: 1.9.0 `f` is now able to return a 1-D array-like, with the :math:`(m, n)` Jacobian being estimated. epsilon : {float, array_like}, optional Increment to `xk` to use for determining the function gradient. If a scalar, uses the same finite difference delta for all partial derivatives. If an array, should contain one value per element of `xk`. Defaults to ``sqrt(np.finfo(float).eps)``, which is approximately 1.49e-08. \*args : args, optional Any other arguments that are to be passed to `f`. Returns ------- jac : ndarray The partial derivatives of `f` to `xk`. See Also -------- check_grad : Check correctness of gradient function against approx_fprime. Notes ----- The function gradient is determined by the forward finite difference formula:: f(xk[i] + epsilon[i]) - f(xk[i]) f'[i] = --------------------------------- epsilon[i] Examples -------- >>> import numpy as np >>> from scipy import optimize >>> def func(x, c0, c1): ... "Coordinate vector `x` should be an array of size two." ... return c0 * x[0]**2 + c1*x[1]**2 >>> x = np.ones(2) >>> c0, c1 = (1, 200) >>> eps = np.sqrt(np.finfo(float).eps) >>> optimize.approx_fprime(x, func, [eps, np.sqrt(200) * eps], c0, c1) array([ 2. , 400.00004208]) r)rjabs_steprFf0)rCrrr%)xkrrrFr;s r?rrsMT B  B 2B Q9w"&2 / / //rAseed) position_numrD)r directionrngc L|}tj|}d}|dkrtj||g|R} | jdkrt dt |} | |j} ||| f|z} |} tjd}tj | | }n(|dkr|} |} |}||g|R}nt |dtj tj tj |t|| |g| Rz d zS) al Check the correctness of a gradient function by comparing it against a (forward) finite-difference approximation of the gradient. Parameters ---------- func : callable ``func(x0, *args)`` Function whose derivative is to be checked. grad : callable ``grad(x0, *args)`` Jacobian of `func`. x0 : ndarray Points to check `grad` against forward difference approximation of grad using `func`. args : \\*args, optional Extra arguments passed to `func` and `grad`. epsilon : float, optional Step size used for the finite difference approximation. It defaults to ``sqrt(np.finfo(float).eps)``, which is approximately 1.49e-08. direction : str, optional If set to ``'random'``, then gradients along a random vector are used to check `grad` against forward difference approximation using `func`. By default it is ``'all'``, in which case, all the one hot direction vectors are considered to check `grad`. If `func` is a vector valued function then only ``'all'`` can be used. rng : `numpy.random.Generator`, optional Pseudorandom number generator state. When `rng` is None, a new `numpy.random.Generator` is created using entropy from the operating system. Types other than `numpy.random.Generator` are passed to `numpy.random.default_rng` to instantiate a ``Generator``. The random numbers generated affect the random vector along which gradients are computed to check ``grad``. Note that `rng` is only used when `direction` argument is set to `'random'`. Returns ------- err : float The square root of the sum of squares (i.e., the 2-norm) of the difference between ``grad(x0, *args)`` and the finite difference approximation of `grad` using func at the points `x0`. See Also -------- approx_fprime Examples -------- >>> import numpy as np >>> def func(x): ... return x[0]**2 - 0.5 * x[1]**3 >>> def grad(x): ... return [2 * x[0], -1.5 * x[1]**2] >>> from scipy.optimize import check_grad >>> check_grad(func, grad, [1.5, -1.5]) 2.9802322387695312e-08 # may vary >>> rng = np.random.default_rng() >>> check_grad(func, grad, [1.5, -1.5], ... direction='random', seed=rng) 2.9802322387695312e-08 c ||||zzg|RSr9rR)wrrvrFs r?gzcheck_grad..gLs"tB1H$t$$$$rArandomr!z1'random' can only be used with scalar valued func)rr!rDz1 is not a valid string for ``direction`` argumentr)rCr asanyarrayrrur(standard_normalrrdotrrrr)rrrrr@rArFsteprF_gradrng_genrE_args_funcvarsanalytical_grads r?rr sq~ D BB%%%H dd2oooo.. :>>%&& &$S))  # #"( # 4 4r1 $x~~&** e  $r/D///I22233 3 726"& =udCUCCC CaG  rAcB||f|z}||||zzf|z}||z |z Sr9rR)rrfprimerrFf1f2s r?approx_fhess_prWhs? 2%$, B 2 >#d* ,B Gw rAceZdZdS)_LineSearchErrorNrorRrAr?rYrYorprArYc |dd}t|||||||fi|} | d2|0|| d|zz} || d| | d| dsd} | dptj5tjdt i} dD]} | |vr || | | <t |||||||fd|i| } dddn #1swxYwY| dt| S) a  Same as line_search_wolfe1, but fall back to line_search_wolfe2 if suitable step length is not found, and raise an exception if a suitable step length is not found. Raises ------ _LineSearchError If no suitable step size is found extra_conditionNrrr9ignore)c1c2r)popr"rcatch_warnings simplefilterr$r#rY) rrTr<pkgfkold_fval old_old_fvalkwargsr[retxp1kwargs2keys r?_line_search_wolfe12rlssjj!2D99O QB%| ' '% ' 'C 1vo93q6B;s1vsCFCF;; C 1v~  $ & & 0 0  !(,= > > >G+ / /&==#)#;GCL$QB%-|005D0(/00C 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1v~   Js?ACCCh㈵>g?c |||| || | | ||d }t| } t||||fd| i|}|rC|d|d|d|d|d|d|d f}| r ||d fz }|S| r|d|d fS|dS) a Minimize a function using the BFGS algorithm. Parameters ---------- f : callable ``f(x,*args)`` Objective function to be minimized. x0 : ndarray Initial guess, shape (n,) fprime : callable ``f'(x,*args)``, optional Gradient of f. args : tuple, optional Extra arguments passed to f and fprime. gtol : float, optional Terminate successfully if gradient norm is less than `gtol` norm : float, optional Order of norm (Inf is max, -Inf is min) epsilon : int or ndarray, optional If `fprime` is approximated, use this value for the step size. callback : callable, optional An optional user-supplied function to call after each iteration. Called as ``callback(xk)``, where ``xk`` is the current parameter vector. maxiter : int, optional Maximum number of iterations to perform. full_output : bool, optional If True, return ``fopt``, ``func_calls``, ``grad_calls``, and ``warnflag`` in addition to ``xopt``. disp : bool, optional Print convergence message if True. retall : bool, optional Return a list of results at each iteration if True. xrtol : float, default: 0 Relative tolerance for `x`. Terminate successfully if step size is less than ``xk * xrtol`` where ``xk`` is the current parameter vector. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. hess_inv0 : None or ndarray, optional`` Initial inverse hessian estimate, shape (n, n). If None (default) then the identity matrix is used. Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float Minimum value. gopt : ndarray Value of gradient at minimum, f'(xopt), which should be near 0. Bopt : ndarray Value of 1/f''(xopt), i.e., the inverse Hessian matrix. func_calls : int Number of function_calls made. grad_calls : int Number of gradient calls made. warnflag : integer 1 : Maximum number of iterations exceeded. 2 : Gradient and/or function calls not changing. 3 : NaN result encountered. allvecs : list The value of `xopt` at each iteration. Only returned if `retall` is True. Notes ----- Optimize the function, `f`, whose gradient is given by `fprime` using the quasi-Newton method of Broyden, Fletcher, Goldfarb, and Shanno (BFGS). Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. See Also -------- minimize: Interface to minimization algorithms for multivariate functions. See ``method='BFGS'`` in particular. References ---------- Wright, and Nocedal 'Numerical Optimization', 1999, p. 198. Examples -------- >>> import numpy as np >>> from scipy.optimize import fmin_bfgs >>> def quadratic_cost(x, Q): ... return x @ Q @ x ... >>> x0 = np.array([-3, -4]) >>> cost_weight = np.diag([1., 10.]) >>> # Note that a trailing comma is necessary for a tuple with single element >>> fmin_bfgs(quadratic_cost, x0, args=(cost_weight,)) Optimization terminated successfully. Current function value: 0.000000 Iterations: 7 # may vary Function evaluations: 24 # may vary Gradient evaluations: 8 # may vary array([ 2.85169950e-06, -4.61820139e-07]) >>> def quadratic_cost_grad(x, Q): ... return 2 * Q @ x ... >>> fmin_bfgs(quadratic_cost, x0, quadratic_cost_grad, args=(cost_weight,)) Optimization terminated successfully. Current function value: 0.000000 Iterations: 7 Function evaluations: 8 Gradient evaluations: 8 array([ 2.85916637e-06, -4.54371951e-07]) ) gtolnormepsrr2rxrtolr^r_ hess_inv0r\r=r:r;hess_invrnjevrr)rl_minimize_bfgs)rrrTrFrorprr2rrrr\rrr^r_rsrr[rs r?rrsj " $ $Dh''H Bf H Hx H4 H HC s8SZUS_v;F S]<  * I) )G  s8S^+ +s8OrAc t|t|| }t|}|jdkrd|_|t |dz}t|||||| }|j}|j }||}||}d}t |}tj |t}||n|}|tj |dz z}|}|r|g}d}t||}||kr||kr tj|| } t#|||| |||d d | | \}!}"}#}}}$n#t$$rd}YnwxYw|!| z}%||%z}&|r||&|&}|$ ||&}$|$|z }'|$}|d z }t)|| }(t+||(rn_t||}||krnF|!t| z| | t|zzkrntj|sd}ntj|'|%})|)dkrd}*| rd}+t/d|+nd|)z }*||%ddtjf|'tjddfz|*zz },||'ddtjf|%tjddfz|*zz }-tj|,tj||-|*|%ddtjfz|%tjddfzz}||kr||k |}.|dkrt2d}+n||krd }t2d}+nktj|s:tj|.s&tj|rd}t2d}+n t2d}+| rct/||+t9d|.dt9d|zt9d|jzt9d|jzt)|.|||j|j||dk|+|| }/|r||/d<|/S)a Minimization of scalar function of one or more variables using the BFGS algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter : int Maximum number of iterations to perform. gtol : float Terminate successfully if gradient norm is less than `gtol`. norm : float Order of norm (Inf is max, -Inf is min). eps : float or ndarray If `jac is None` the absolute step size used for numerical approximation of the jacobian via forward differences. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x) * max(1, abs(x))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. xrtol : float, default: 0 Relative tolerance for `x`. Terminate successfully if step size is less than ``xk * xrtol`` where ``xk`` is the current parameter vector. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. hess_inv0 : None or ndarray, optional Initial inverse hessian estimate, shape (n, n). If None (default) then the identity matrix is used. Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. If minimization doesn't complete successfully, with an error message of ``Desired error not necessarily achieved due to precision loss``, then consider setting `gtol` to a higher value. This precision loss typically occurs when the (finite difference) numerical differentiation cannot provide sufficient precision to satisfy the `gtol` termination criterion. This can happen when working in single precision and a callable jac is not provided. For single precision problems a `gtol` of 1e-3 seems to work. rrHNr)rFrrrrr0.++}Ô%IT)rrr^r_r!rg@@z.Divide-by-zero encountered: rhok assumed largeTrr3r2rr4r0rrrr! Gradient evaluations: %d) r:r;rtrrurr0rr=rr)rrwrrrrrrr:rrCrintlinalgrprrKrlrYrrr*r_print_success_message_or_warnnewaxisrisnanrrrngev)0r:rrFr;r\rorprqr2rrrrrr^r_rsrrrrmyfprimererdr#r!Irvrfr<rr7gnormrcalpha_kfcgcgfkp1skxkp1ykrXrhok_invrhokrA1A2r6r8s0 r?rvrv0sn?+++Y''' F     B w!||b''C- !#r3T37K M M MB AwHquuH (2,,C A BA qAYBbinnS11A55L B $H CT " " "E 4< > > {8$$ H 6"b>> r>>D :F.tS999=D AAArzM"R AAA %66= = AAArzM"R AAA %66= = VBr2 ' '4"QQQ ]2C+C13BJM1B,CDu 4<.>.@.@)e$i( =&x555 :$:::;;; '!+,,, 1BG;<<< 1BG;<<< #!#%-]SB !###F$#y Ms7!E E)(E)ch|st|dStj||ptddS)Nrrz)rrrr)r7rwarntypes r?rrs< J g gx:?qIIIIIIrAg?c|||| || d}t| } t||||f| | | d|}|r5|d|d|d|d|df}| r ||dfz }|S| r|d|dfS|dS) a: Minimize a function using a nonlinear conjugate gradient algorithm. Parameters ---------- f : callable, ``f(x, *args)`` Objective function to be minimized. Here `x` must be a 1-D array of the variables that are to be changed in the search for a minimum, and `args` are the other (fixed) parameters of `f`. x0 : ndarray A user-supplied initial estimate of `xopt`, the optimal value of `x`. It must be a 1-D array of values. fprime : callable, ``fprime(x, *args)``, optional A function that returns the gradient of `f` at `x`. Here `x` and `args` are as described above for `f`. The returned value must be a 1-D array. Defaults to None, in which case the gradient is approximated numerically (see `epsilon`, below). args : tuple, optional Parameter values passed to `f` and `fprime`. Must be supplied whenever additional fixed parameters are needed to completely specify the functions `f` and `fprime`. gtol : float, optional Stop when the norm of the gradient is less than `gtol`. norm : float, optional Order to use for the norm of the gradient (``-np.inf`` is min, ``np.inf`` is max). epsilon : float or ndarray, optional Step size(s) to use when `fprime` is approximated numerically. Can be a scalar or a 1-D array. Defaults to ``sqrt(eps)``, with eps the floating point machine precision. Usually ``sqrt(eps)`` is about 1.5e-8. maxiter : int, optional Maximum number of iterations to perform. Default is ``200 * len(x0)``. full_output : bool, optional If True, return `fopt`, `func_calls`, `grad_calls`, and `warnflag` in addition to `xopt`. See the Returns section below for additional information on optional return values. disp : bool, optional If True, return a convergence message, followed by `xopt`. retall : bool, optional If True, add to the returned values the results of each iteration. callback : callable, optional An optional user-supplied function, called after each iteration. Called as ``callback(xk)``, where ``xk`` is the current value of `x0`. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.4 Parameter for curvature condition rule. Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float, optional Minimum value found, f(xopt). Only returned if `full_output` is True. func_calls : int, optional The number of function_calls made. Only returned if `full_output` is True. grad_calls : int, optional The number of gradient calls made. Only returned if `full_output` is True. warnflag : int, optional Integer value with warning status, only returned if `full_output` is True. 0 : Success. 1 : The maximum number of iterations was exceeded. 2 : Gradient and/or function calls were not changing. May indicate that precision was lost, i.e., the routine did not converge. 3 : NaN result encountered. allvecs : list of ndarray, optional List of arrays, containing the results at each iteration. Only returned if `retall` is True. See Also -------- minimize : common interface to all `scipy.optimize` algorithms for unconstrained and constrained minimization of multivariate functions. It provides an alternative way to call ``fmin_cg``, by specifying ``method='CG'``. Notes ----- This conjugate gradient algorithm is based on that of Polak and Ribiere [1]_. Conjugate gradient methods tend to work better when: 1. `f` has a unique global minimizing point, and no local minima or other stationary points, 2. `f` is, at least locally, reasonably well approximated by a quadratic function of the variables, 3. `f` is continuous and has a continuous gradient, 4. `fprime` is not too large, e.g., has a norm less than 1000, 5. The initial guess, `x0`, is reasonably close to `f` 's global minimizing point, `xopt`. Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. References ---------- .. [1] Wright & Nocedal, "Numerical Optimization", 1999, pp. 120-122. Examples -------- Example 1: seek the minimum value of the expression ``a*u**2 + b*u*v + c*v**2 + d*u + e*v + f`` for given values of the parameters and an initial guess ``(u, v) = (0, 0)``. >>> import numpy as np >>> args = (2, 3, 7, 8, 9, 10) # parameter values >>> def f(x, *args): ... u, v = x ... a, b, c, d, e, f = args ... return a*u**2 + b*u*v + c*v**2 + d*u + e*v + f >>> def gradf(x, *args): ... u, v = x ... a, b, c, d, e, f = args ... gu = 2*a*u + b*v + d # u-component of the gradient ... gv = b*u + 2*c*v + e # v-component of the gradient ... return np.asarray((gu, gv)) >>> x0 = np.asarray((0, 0)) # Initial guess. >>> from scipy import optimize >>> res1 = optimize.fmin_cg(f, x0, fprime=gradf, args=args) Optimization terminated successfully. Current function value: 1.617021 Iterations: 4 Function evaluations: 8 Gradient evaluations: 8 >>> res1 array([-1.80851064, -0.25531915]) Example 2: solve the same problem using the `minimize` function. (This `myopts` dictionary shows all of the available options, although in practice only non-default values would be needed. The returned value will be a dictionary.) >>> opts = {'maxiter' : None, # default value. ... 'disp' : True, # non-default value. ... 'gtol' : 1e-5, # default value. ... 'norm' : np.inf, # default value. ... 'eps' : 1.4901161193847656e-08} # default value. >>> res2 = optimize.minimize(f, x0, jac=gradf, args=args, ... method='CG', options=opts) Optimization terminated successfully. Current function value: 1.617021 Iterations: 4 Function evaluations: 8 Gradient evaluations: 8 >>> res2.x # minimum found array([-1.80851064, -0.25531915]) )rorprqrr2rr\r^r_r=r:rrurr)rl _minimize_cg)rrrTrFrorprr2rrrr\r^r_rr[rs r?rrs@  " "Dh''H q"dF X"    C c(CJF S[#h-O  * I) )G  s8S^+ +s8OrAc !"#$%&'(t|| }t|}|t|dz}t |||||| }|j}|j$||}$|#d}|(|tj #dz z}|r(g}d}# %t#}d'|kr||krtj ##"dg!d"#$%(fd &!&'fd } t|$(%#||| | d d | \}}}}}}n#t$rd}YnuwxYw|!dkr !\}(%#}n&||\}(%#}|r|(|d z }t(|}t!||rn |kr||k|}|dkrt"d}n||krd }t"d}nktj|s:tj|s&tj(rd}t"d}n t"d}| rct)||t+d|dt+d|zt+d|jzt+d|jzt|#|j|j||dk|(| } |r|| d<| S)a Minimization of scalar function of one or more variables using the conjugate gradient algorithm. Options ------- disp : bool Set to True to print convergence messages. maxiter : int Maximum number of iterations to perform. gtol : float Gradient norm must be less than `gtol` before successful termination. norm : float Order of norm (Inf is max, -Inf is min). eps : float or ndarray If `jac is None` the absolute step size used for numerical approximation of the jacobian via forward differences. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. finite_diff_rel_step : None or array_like, optional If ``jac in ['2-point', '3-point', 'cs']`` the relative step size to use for numerical approximation of the jacobian. The absolute step size is computed as ``h = rel_step * sign(x) * max(1, abs(x))``, possibly adjusted to fit into the bounds. For ``jac='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.4 Parameter for curvature condition rule. Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. Nr)r;rFrrrrrxg{Gz?c | zz}|  |}|z }tdtj||z }| | zz}t| }|||||fS)Nrrx)rrCrKr) alpharrrbeta_kpkp1rdeltakrdrrprcr<s r?polak_ribiere_powell_stepz/_minimize_cg..polak_ribiere_powell_steps~ ?D} BBF2u--677F6FRK'DEt,,,E4ue4 4rAc ||dd<\}}}}}| krdStj|| tj||zkS)NT)rCrK) rrfp1rr<rcrdr cached_steprorsigma_3s r?descent_conditionz'_minimize_cg..descent_conditionsi76ueDDKN(3 %E2r3}}t6"c??whS1A1A&AA ArAryrz)r^r_rrr[r!rr3r2rr4r0rrrrr|) r:r;rrurr0rr=rrr9)rrrrrr:rrCr~rprrKrlrYrrr*rrrrrrr))r:rrFr;r\rorprqr2rrrr^r_rrrrrer#rfrr7rrrrrrrXr6rr8rrrdrrcrrr<s) `` @@@@@@@@r?rrsR?+++ F     Bb''C- !#rss7K M M MB AwHquuH (2,,C A BbinnS11A55L $H B CT " " "EG 4<.>.@.@)e$i( =&x555 :$:::;;; '!+,,, 1BG;<<< 1BG;<<< #BG!#%-]SB !###F$#y Ms"D++ D:9D:c |||| | d}t| } t||||||f| | |d|}| r<|d|d|d|d|d|df}| r ||d fz }|S| r|d|d fS|dS) a Unconstrained minimization of a function using the Newton-CG method. Parameters ---------- f : callable ``f(x, *args)`` Objective function to be minimized. x0 : ndarray Initial guess. fprime : callable ``f'(x, *args)`` Gradient of f. fhess_p : callable ``fhess_p(x, p, *args)``, optional Function which computes the Hessian of f times an arbitrary vector, p. fhess : callable ``fhess(x, *args)``, optional Function to compute the Hessian matrix of f. args : tuple, optional Extra arguments passed to f, fprime, fhess_p, and fhess (the same set of extra arguments is supplied to all of these functions). epsilon : float or ndarray, optional If fhess is approximated, use this value for the step size. callback : callable, optional An optional user-supplied function which is called after each iteration. Called as callback(xk), where xk is the current parameter vector. avextol : float, optional Convergence is assumed when the average relative error in the minimizer falls below this amount. maxiter : int, optional Maximum number of iterations to perform. full_output : bool, optional If True, return the optional outputs. disp : bool, optional If True, print convergence message. retall : bool, optional If True, return a list of results at each iteration. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule Returns ------- xopt : ndarray Parameters which minimize f, i.e., ``f(xopt) == fopt``. fopt : float Value of the function at xopt, i.e., ``fopt = f(xopt)``. fcalls : int Number of function calls made. gcalls : int Number of gradient calls made. hcalls : int Number of Hessian calls made. warnflag : int Warnings generated by the algorithm. 1 : Maximum number of iterations exceeded. 2 : Line search failure (precision loss). 3 : NaN result encountered. allvecs : list The result at each iteration, if retall is True (see below). See also -------- minimize: Interface to minimization algorithms for multivariate functions. See the 'Newton-CG' `method` in particular. Notes ----- Only one of `fhess_p` or `fhess` need to be given. If `fhess` is provided, then `fhess_p` will be ignored. If neither `fhess` nor `fhess_p` is provided, then the hessian product will be approximated using finite differences on `fprime`. `fhess_p` must compute the hessian times an arbitrary vector. If it is not given, finite-differences on `fprime` are used to compute it. Newton-CG methods are also called truncated Newton methods. This function differs from scipy.optimize.fmin_tnc because 1. scipy.optimize.fmin_ncg is written purely in Python using NumPy and scipy while scipy.optimize.fmin_tnc calls a C function. 2. scipy.optimize.fmin_ncg is only for unconstrained minimization while scipy.optimize.fmin_tnc is for unconstrained minimization or box constrained minimization. (Box constraints give lower and upper bounds for each variable separately.) Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. References ---------- Wright & Nocedal, 'Numerical Optimization', 1999, p. 140. )rrqr2rrrr=r:rrunhevrr)rl_minimize_newtoncg)rrrTfhess_pfhessrFavextolrr2rrrr\r^r_rr[rs r?rr9sB  " "D h''H QD&% F&.2" F F@D F FC s8SZVc&kv;H /  * I) )G  s8S^+ +s8OrAc 8 456789:;t||td|}|}|}|}| 9t|}t ||||||::j}:j}:|}|tvst|tr d}:fd}|}4 56789:;f d}d6| t|dz} dt|z}t||z}tj tj}tj|;9r;g4d7d5||8d}tj tjj}||kr׉7| krd t&d z}|d |S|; }tj|d }t-d t/j|}||z} t3t||j}!| }"|" }#d}$tj|"|"}%|:;}&6d z 6t9|D]}'tjtj|"| krn|%|tA;|#||}(n%|;|#g|R}(6d z 6n|&|#}(t|(!}(tj|#|(})d|)cxkr d|zkrnnnh|)dkr|$dkrnZ|%|) z |z}!nO|%|)z }*|!|*|#zz }!|"|*|(zz }"tj|"|"}+|+|%z },|" |,|#zz}#|$d z }$|+}%d}|d|S|!}-| 5 tE||;|-58|| |  \}.}/}08}}1n,#tF$rd t&dz}|d|cYSwxYw|.|-z}2;|2z ;9r4$;7d z 7tK;8}3tM||3r |ddStj|2d }||ktj'8stj'|r|dt&dSt&d}|d|S)ap Minimization of scalar function of one or more variables using the Newton-CG algorithm. Note that the `jac` parameter (Jacobian) is required. Options ------- disp : bool Set to True to print convergence messages. xtol : float Average relative error in solution `xopt` acceptable for convergence. maxiter : int Maximum number of iterations to perform. eps : float or ndarray If `hessp` is approximated, use this value for the step size. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. c1 : float, default: 1e-4 Parameter for Armijo condition rule. c2 : float, default: 0.9 Parameter for curvature condition rule. Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. Nz)Jacobian is required for Newton-CG method)rFrrcT||Sr9)rrK)r=rrFrs r?_hesspz"_minimize_newtoncg.._hessps771::>>!$$ $rAc V rut||td dtdztd jztd jztdz }t | j j||dk|  } r|d <|S) Nrrrrr|z Hessian evaluations: %dr) r:r;rrurrr0rr=rr)rrrrr) r7rr6r8rrrdhcallsr#rerrr<s r? terminatez%_minimize_newtoncg..terminates  ? *8S 9 9 9 BhBBB C C C +a/ 0 0 0 5? @ @ @ 5? @ @ @ 4v= > > >Dc%'W6()1Q$%'''  ( 'F9  rArrz Warning: r2r!rxrrrzMWarning: CG iterations didn't converge. The Hessian is not positive definite.)r^r_r3rrr\r4r0)(rrurrrr:rrr- isinstancer rrCfinforrrErrqrr~rprmathrrrrKrrrrrWsqueezerlrYrrr*r)>D4KEC%%-'E FADD(C9Q$$ $ b %)!VRS*2LRBPPP :FBHlEE  % % % ::C9Q$$ $ $ $ % " f    NN2    Q,rx@@@ $X/B C C $9Q## # v155 _ $  d 8H   8-!8!8 89Q 677 7i(yC   s=N&OOc|||d}t|||f|fi|} |r| d| d| d| dfS| dS)a Bounded minimization for scalar functions. Parameters ---------- func : callable f(x,*args) Objective function to be minimized (must accept and return scalars). x1, x2 : float or array scalar Finite optimization bounds. args : tuple, optional Extra arguments passed to function. xtol : float, optional The convergence tolerance. maxfun : int, optional Maximum number of function evaluations allowed. full_output : bool, optional If True, return optional outputs. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. Returns ------- xopt : ndarray Parameters (over given interval) which minimize the objective function. fval : number (Optional output) The function value evaluated at the minimizer. ierr : int (Optional output) An error flag (0 if converged, 1 if maximum number of function calls reached). numfunc : int (Optional output) The number of function calls made. See also -------- minimize_scalar: Interface to minimization algorithms for scalar univariate functions. See the 'Bounded' `method` in particular. Notes ----- Finds a local minimizer of the scalar function `func` in the interval x1 < xopt < x2 using Brent's method. (See `brent` for auto-bracketing.) References ---------- .. [1] Forsythe, G.E., M. A. Malcolm, and C. B. Moler. "Computer Methods for Mathematical Computations." Prentice-Hall Series in Automatic Computation 259 (1977). .. [2] Brent, Richard P. Algorithms for Minimization Without Derivatives. Courier Corporation, 2013. Examples -------- `fminbound` finds the minimizer of the function in the given range. The following examples illustrate this. >>> from scipy import optimize >>> def f(x): ... return (x-1)**2 >>> minimizer = optimize.fminbound(f, -4, 4) >>> minimizer 1.0 >>> minimum = f(minimizer) >>> minimum 0.0 >>> res = optimize.fminbound(f, 3, 4, full_output=True) >>> minimizer, fval, ierr, numfunc = res >>> minimizer 3.000005960860986 >>> minimum = f(minimizer) >>> minimum, fval (4.000023843479476, 4.000023843479476) )rr2rr=r:rr)_minimize_scalar_bounded) rx1x2rFrrrroptionsr[s r?rrmsif G #4"b4 C C7 C CC3xUS]CK??3xrAc vt||}t|dkrtd|\}} t|rt| std|| krtdd} d} d} t d} d d t d z z}|| }}||||z zz}||}}d x}}|}||g|R}d }d ||f}t j}|x}}d ||zz}| t j|z|d z z}d|z}|dkr4tdt| td|| fzzt j||z |d ||z zz kr;d } t j||krd} ||z ||z z}!||z ||z z}"||z |"z||z |!zz }#d|"|!z z}"|"d kr|# }#t j|"}"|}!|}t j|#t jd |"z|!zkr`|#|"||z zkrT|#|"||z zkrH|#d z|"z }||z}d} ||z |ks ||z |kr&t j ||z ||z dkz}$||$z}nd } | r||kr||z }n||z }||z}d} t j ||dkz}$||$t j t j||zz}||g|R}|d z }|||f}|dkrtd|| fzz||kr||kr|}n|}||}}||}}||}}n6||kr|}n|}||ks||kr ||}}||}}n||ks ||ks||kr||}}d ||zz}| t j|z|d z z}d|z}||krd } n%t j||z |d ||z zz k;t j |s(t j |st j |rd} |}%|dkrt|| |%|||t|%| | dkddtdd| d|||}&|&S)a Options ------- maxiter : int Maximum number of iterations to perform. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. xatol : float Absolute error in solution `xopt` acceptable for convergence. rzbounds must have two elements.z+Optimization bounds must be finite scalars.z(The lower bound exceeds the upper bound.rz2 Func-count x f(x) Procedurez initialgOd@>a"f%%3w!mH)UdNNQ$b))b2g!^%@%@ADD"  s 555F MrAc6eZdZ d dZd dZdZd Zdd ZdS)BrentrR`sbO>rrc||_||_||_||_d|_d|_d|_d|_d|_d|_ ||_ dS)Ngdy=gŋ!r?r) rrFtolr2_mintol_cgxminr6iterfuncallsr)r>rrFrr2rrs r?r@zBrent.__init__c sS         rANc||_dSr9)brack)r>rs r? set_bracketzBrent.set_bracketr s  rAc|j}|j}|j}|t||\}}}}}} } nt |dkr(t||d|d|\}}}}}} } nt |dkrd|\}}}||kr||}}||kr||kst d||f|z}||f|z}||f|z} ||kr|| kst dd} nt d |||||| | fS) NrFrrr!xaxbrFrWBracketing values (xa, xb, xc) do not fulfill this requirement: (xa < xb) and (xb < xc)cBracketing values (xa, xb, xc) do not fulfill this requirement: (f(xb) < f(xa)) and (f(xb) < f(xc))3Bracketing interval must be length 2 or 3 sequence.)rrFrr rru) r>rrFrrrr1fafbrrs r?get_bracket_infozBrent.get_bracket_infou svyy  =/6t$/G/G/G ,BBBHH ZZ1__/6ta:?(0O0O0O ,BBBHH ZZ1__JBBRRB"WW277 I &B &B &B"WW277 M HH788 82r2r2x//rAc|j}|\}}}}}}}|j} |j} |x} x} } |x}x}}||kr|}|}n|}|}d}d}|jdkrCt dt dddddddd t |d d| d d|d ||jkr*|jtj | z| z}d |z}d ||zz}tj | |z |d ||z zz krntj ||kr| |kr|| z }n|| z }| |z}n| | z ||z z}| | z ||z z}| | z |z| | z |zz }d ||z z}|dkr| }tj |}|}|}|||| z zkrk|||| z zkr_tj |tj d |z|zkr/|dz|z }| |z}||z |ks ||z |kr|| z dkr|}n| }n| |kr|| z }n|| z }| |z}tj ||kr|dkr| |z}n | |z }n| |z}||f|j z}|dz }||kr7|| kr|}n|}||ks| | kr | } |} |}|}n.||ks | | ks| | kr|} |}n|| kr| }n| }| } | } |} |}|}|}|jdkrt |d d| d d|d |dz }||jk*| |_ ||_ ||_ ||_dS)Nr{rrr Func-count^12r=f(x) ^12^12g^12.6grrrr!)rrrrrrr2rrCrrFrr6rr)r>rrrr1rrrrrrr=rDrEfwfvrrrdeltaxrrrxmidrtmp1tmp2rdx_tempurs r?optimizezBrent.optimize s(y+/+@+@+B+B(BBB,hAR" GGAAAAA 9q== #JJJ \??????&??? @ @ @ X;;;Q;;;;;; < < <dl""8bfQii''1D:D!a%=Dva$h4#Q-#788v$&&IIUFFUFFlA"r'*A"r'*UdNa!et^3dTk*3JJAvd|| Q''a$!a%..@.@RVC$J,@%A%AAAc'D.CCAQ$1q5D..!8q=="&CC#'%CT !"Q!"Q,Cs d""!88DAADAAGty(*B MHREEAAA"HH!q&&AABBBBhhAFFQABFFAAAy1}}??????B???@@@ AIDcdl""l     rAFcH|r|j|j|j|jfS|jSr9)rr6rr)r>rs r? get_resultzBrent.get_result s)  9diDMA A9 rA)rRrrrrr9)F)rNrOrPr@rrrrrRrAr?rra sy;>%&    $0$0$0Lr!r!r!hrArrcz||d}t|||fi|}|r|d|d|d|dfS|dS)a Given a function of one variable and a possible bracket, return a local minimizer of the function isolated to a fractional precision of tol. Parameters ---------- func : callable f(x,*args) Objective function. args : tuple, optional Additional arguments (if present). brack : tuple, optional Either a triple ``(xa, xb, xc)`` satisfying ``xa < xb < xc`` and ``func(xb) < func(xa) and func(xb) < func(xc)``, or a pair ``(xa, xb)`` to be used as initial points for a downhill bracket search (see `scipy.optimize.bracket`). The minimizer ``x`` will not necessarily satisfy ``xa <= x <= xb``. tol : float, optional Relative error in solution `xopt` acceptable for convergence. full_output : bool, optional If True, return all output args (xmin, fval, iter, funcalls). maxiter : int, optional Maximum number of iterations in solution. Returns ------- xmin : ndarray Optimum point. fval : float (Optional output) Optimum function value. iter : int (Optional output) Number of iterations. funcalls : int (Optional output) Number of objective function evaluations made. See also -------- minimize_scalar: Interface to minimization algorithms for scalar univariate functions. See the 'Brent' `method` in particular. Notes ----- Uses inverse parabolic interpolation when possible to speed up convergence of golden section method. Does not ensure that the minimum lies in the range specified by `brack`. See `scipy.optimize.fminbound`. Examples -------- We illustrate the behaviour of the function when `brack` is of size 2 and 3 respectively. In the case where `brack` is of the form ``(xa, xb)``, we can see for the given values, the output does not necessarily lie in the range ``(xa, xb)``. >>> def f(x): ... return (x-1)**2 >>> from scipy import optimize >>> minimizer = optimize.brent(f, brack=(1, 2)) >>> minimizer 1 >>> res = optimize.brent(f, brack=(-1, 0.5, 2), full_output=True) >>> xmin, fval, iter, funcalls = res >>> f(xmin), fval (0.0, 0.0) rr2r=r:rr)_minimize_scalar_brentrrFrrrr2rr[s r?rr s`N!##G ud > >g > >C3xUSZV<<3xrAc Bt||}|dkrtd|t|||d||}||||d\} } } } | |ko(t j| pt j|  } | rd|d}n>| |krd}t j| st j| rtd }|rt| |t| | | | | | S) a: Options ------- maxiter : int Maximum number of iterations to perform. xtol : float Relative error in solution `xopt` acceptable for convergence. disp : int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. Notes ----- Uses inverse parabolic interpolation when possible to speed up convergence of golden section method. rztolerance should be >= 0, got T)rrFrrr2r)rk Optimization terminated successfully; The returned value satisfies the termination criteria (using xtol =  )& Maximum number of iterations exceededr4)r:r=rrr0r) rrurrrrrCrrrr)rrrFrr2rrrrr=r6rrr0rs r?rrf sa6?+++ C QwwA#AABBB t$C"G$ @ @ @E e NN))d);;AtS$GmCRXa[[%BBHTNN CG2-$(--- '>>?G 8A;; 2"(4.. 2(/1G =&7{G<<< daSt")7 < < <>> def f(x): ... return (x-1)**2 >>> from scipy import optimize >>> minimizer = optimize.golden(f, brack=(1, 2)) >>> minimizer 1 >>> res = optimize.golden(f, brack=(-1, 0.5, 2), full_output=True) >>> xmin, fval, funcalls = res >>> f(xmin), fval (9.925165290385052e-18, 9.925165290385052e-18) rr=r:r)_minimize_scalar_goldenrs r?r r sUJw//G !$t ? ?w ? ?C3xUS[003xrAc t||}|t||\}} } } } } }nt|dkr(t||d|d|\}} } } } } }nt|dkrd|\}} } || kr|| }} || kr| | kstd||f|z} || f|z} || f|z} | | kr| | kstd d}ntd d }d |z }| }|}t j| | z t j| |z kr| }| || | z zz}n | }| || |z zz }||f|z}||f|z}|dz }d}|dkr)t d t ddd ddd ddt|D]}t j||z |t j|t j|zzkrnu||kr|}|}||z||zz}|}||f|z}n|}|}||z||zz}|}||f|z}|dz }|dkr)||kr||}}n||}}t |dd |dd |d|dz }||kr|}|}n|}|}||ko(t j|pt j| }|rd|d}n>||krd}t j|st j|rtd}|rt| |t||||||S)a Options ------- xtol : float Relative error in solution `xopt` acceptable for convergence. maxiter : int Maximum number of iterations to perform. disp: int, optional If non-zero, print messages. ``0`` : no message printing. ``1`` : non-convergence notification messages only. ``2`` : print a message on convergence too. ``3`` : print iteration results. Nrrrr!rrrrrgz7?rrrrr=rrrrrrrr4r:rr=rr0r) rr rrurCrrrrrrr)rrrFrr2rrrrrr1rrrr_gR_gCx3rrrrUrVrrrr6r0rs r?rr s:*?+++ C }+24d+C+C+C(BBB Uq+24E!H6;AhT,K,K,K(BBB Uq B GGBbrBwwE TREDL " TREDL " TREDL "brBwwI NOOO C )C B B rBw"&b//))  #b/ !  #b/ !  B  B MH C axx c  ;;;C;;;f;;;<<< 7^^ 6"r'??cRVBZZ"&**%<= = = E GGBBrC"H$BB &BBBBrC"H$BB &BA  !88Rdd X@@@T@@@4@@@ A A A q RGmFRXd^^%Erx~~ FG2-$(--- '>>?G 8D>> 2RXd^^ 2(/1G =&7{G<<< dTs")7 < < <>> import numpy as np >>> import matplotlib.pyplot as plt >>> from scipy.optimize import bracket >>> def f(x): ... return 10*x**2 + 3*x + 5 >>> x = np.linspace(-2, 2) >>> y = f(x) >>> init_xa, init_xb = 0.1, 1 >>> xa, xb, xc, fa, fb, fc, funcalls = bracket(f, xa=init_xa, xb=init_xb) >>> plt.axvline(x=init_xa, color="k", linestyle="--") >>> plt.axvline(x=init_xb, color="k", linestyle="--") >>> plt.plot(x, y, "-k") >>> plt.plot(xa, fa, "bx") >>> plt.plot(xb, fb, "rx") >>> plt.plot(xc, fc, "bx") >>> plt.show() Note that both initial points were to the right of the minimum, and the third point was found in the "downhill" direction: the direction in which the function appeared to be decreasing (to the left). The final points are strictly ordered, and the function value at the middle point is less than the function values at the endpoints; it follows that a minimum must lie within the bracket. gw?gO ;rrrzNo valid bracket was found before the iteration limit was reached. Consider trying different initial points or increasing `maxiter`.r!r{zcThe algorithm terminated without finding a valid bracket. Consider trying different initial points.)rCrr RuntimeErrorr BracketErrordata)rrrrF grow_limitr2_gold_verysmall_numrrr1rrrr r valdenomrDwlimrrcond1cond2cond3rs r?r r d sT EN ZR ! !FB ut| B ut| B RRBRB erBw B  BH D 77RBG$RBG$Tk 6#;; ' '.(EE#IE 27d"b2g%55> >J"r'**' '>>s## #   FrAv  $ $t %B MHRr''Ub2g&&At %B MHH$h #s * *At %B MHH$ha 3 & &t %B MHR"r'**TQD4K)A Ub2g&&At %B MH      q 77v"W !r .myfunc} stAbL!!!rAN)rrr!)r6rd)rc>tj|Sr9)rCtan)r=rGs r?z$_linesearch_powell.. s&&++rA)rCrr6rtupler=r:rBisneginfisposinf_linesearch_powellrarctanrJ) rrrFrrr r6r[ alpha_minfretboundrGs ``` @r?rOrOh s*""""""" 6"::'!%!1q" QB7GH  !4)*@*0$cKKK%4 ^QVR B [AA ;uQx  'R[q%:%: '%dAr#FFF FU1X&& 'r{58/D/D '*65c JJJCB7AFB& & IeAh''58)<)<>> def f(x): ... return x**2 >>> from scipy import optimize >>> minimum = optimize.fmin_powell(f, -1) Optimization terminated successfully. Current function value: 0.000000 Iterations: 2 Function evaluations: 16 >>> minimum array(0.0) )rrr2r1rdirecrr\r=r:rUrrrr)rl_minimize_powell)rrrFrrr2rrrrr\rUrr[rs r?rr sl  " "Dh''H 4T E EH E E EC s8SZWs5zv;H /  * I) )G  s8S^+ +s8OrAc t| |} | }t|}|r|g}t|}| | |dz}|dz} nI|#| tjkr|dz}n1tj}n$| "|tjkr|dz} n tj} t ||| \}}| t|t} n`t| t} tj | | j dkrtj dtd|d\}}nZ|j|j}}t j||kst j||krtj d td||}|}d} |}d}d }t'|D]9}| |}|}t)||||d z||| \}}}||z |kr||z }|}:|dz }|r||t-||}t/||rn`|t j|t j|zzdz}d||z z|krn#|d| krn||krn t j|rt j|rn||z }|}||d} nt5||||\}!} |t7| d|zz}"||"}||krud||zd|zz z}#||z |z }$|#|$|$zz}#||z }$|#||$z|$zz}#|#d krAt)||||d z||| \}}}t j|r| d| |<|| d<n#t8$rYnwxYwd}%t:d}&|r@t j||kst j||krd}%t:d}&n{|d| krd}%t:d}&n_||krd}%t:d}&nIt j|s&t j|rd}%t:d}&| rSt=|%|&t>tAd|dtAd|ztAd|dzt-|| ||d|%|%dk|&|}'|r||'d<|'S) a Minimization of scalar function of one or more variables using the modified Powell algorithm. Parameters ---------- fun : callable The objective function to be minimized:: fun(x, *args) -> float where ``x`` is a 1-D array with shape (n,) and ``args`` is a tuple of the fixed parameters needed to completely specify the function. x0 : ndarray, shape (n,) Initial guess. Array of real elements of size (n,), where ``n`` is the number of independent variables. args : tuple, optional Extra arguments passed to the objective function and its derivatives (`fun`, `jac` and `hess` functions). method : str or callable, optional The present documentation is specific to ``method='powell'``, but other options are available. See documentation for `scipy.optimize.minimize`. bounds : sequence or `Bounds`, optional Bounds on decision variables. There are two ways to specify the bounds: 1. Instance of `Bounds` class. 2. Sequence of ``(min, max)`` pairs for each element in `x`. None is used to specify no bound. If bounds are not provided, then an unbounded line search will be used. If bounds are provided and the initial guess is within the bounds, then every function evaluation throughout the minimization procedure will be within the bounds. If bounds are provided, the initial guess is outside the bounds, and `direc` is full rank (or left to default), then some function evaluations during the first iteration may be outside the bounds, but every function evaluation after the first iteration will be within the bounds. If `direc` is not full rank, then some parameters may not be optimized and the solution is not guaranteed to be within the bounds. options : dict, optional A dictionary of solver options. All methods accept the following generic options: maxiter : int Maximum number of iterations to perform. Depending on the method each iteration may use several function evaluations. disp : bool Set to True to print convergence messages. See method-specific options for ``method='powell'`` below. callback : callable, optional Called after each iteration. The signature is:: callback(xk) where ``xk`` is the current parameter vector. Returns ------- res : OptimizeResult The optimization result represented as a ``OptimizeResult`` object. Important attributes are: ``x`` the solution array, ``success`` a Boolean flag indicating if the optimizer exited successfully and ``message`` which describes the cause of the termination. See `OptimizeResult` for a description of other attributes. Options ------- disp : bool Set to True to print convergence messages. xtol : float Relative error in solution `xopt` acceptable for convergence. ftol : float Relative error in ``fun(xopt)`` acceptable for convergence. maxiter, maxfev : int Maximum allowed number of iterations and function evaluations. Will default to ``N*1000``, where ``N`` is the number of variables, if neither `maxiter` or `maxfev` is set. If both `maxiter` and `maxfev` are set, minimization will stop at the first reached. direc : ndarray Initial set of direction vectors for the Powell method. return_all : bool, optional Set to True to return a list of the best solution at each of the iterations. Nr"rrzBdirec input is not full rank, some parameters may not be optimizedrrz)NNrTr{rH)rrr r6r!rg#B ;rrr0ryr5r1rr2r4rrrr)r:rUrrrr0rr=r)!rrrrrCrrrrr~ matrix_rankrrrrrrrrErrOrrr*rrrBrrrrrr)(rrrFr\rrrr2r1rrUrrrrr=rr!r(rr r6rrrbiginddeltardirec1fx2rXbndrA_rttempr7rr8s( r?rVrV/ sx?+++ F F A # AA6>d(T  RV  $hGGfGG  bf  XFFVF;4vNNLFD }AU###U+++ 9  ' '5;q> 9 9 M-)a 9 9 9 9~ $. [[$*9fi[ 6+" # # 9rvb;.>'?'? 9 ML)a 9 9 9 9 477D B D>= BFE1XX  q"4T1f9=ALAL:> #@#@#@a $J%''$JEF AID "q!!!"01$"?"?"? (3FGG "&**rvd||34u@H ) $C  %26+/** %bfQ_.E.E %o. f  h' i( $%28A;;??,,%e$ ?&xnEEE :$:::;;; '$./// 1F1I=>>> Et&)#+h!m$'1...F$#y Ms23B N;N> N N(N=CN NNc|dkr|dkrtd|ddS|dkrd}n|dkrdtd}t||dS) Nrr!r)zO Maximum number of function evaluations exceeded --- increase maxfun argument. r r4)rrr)r=rr6rrrrs r?rr!sy qyy !88 #$(# / / /  qyy- +?5)++"4--- FrArc dt|}|dkrtdt|} t|D]q} t | | t sTt| | dkr)t | | t|fz| | <t | | | | <r|dkr| d} tj | } | j } |dkr`). Supply `-1` to use all cores available to the Process. Alternatively supply a map-like callable, such as `multiprocessing.Pool.map` for evaluating the grid in parallel. This evaluation is carried out as ``workers(func, iterable)``. Requires that `func` be pickleable. .. versionadded:: 1.3.0 Returns ------- x0 : ndarray A 1-D array containing the coordinates of a point at which the objective function had its minimum value. (See `Note 1` for which point is returned.) fval : float Function value at the point `x0`. (Returned when `full_output` is True.) grid : tuple Representation of the evaluation grid. It has the same length as `x0`. (Returned when `full_output` is True.) Jout : ndarray Function values at each point of the evaluation grid, i.e., ``Jout = func(*grid)``. (Returned when `full_output` is True.) See Also -------- basinhopping, differential_evolution Notes ----- *Note 1*: The program finds the gridpoint at which the lowest value of the objective function occurs. If `finish` is None, that is the point returned. When the global minimum occurs within (or not very far outside) the grid's boundaries, and the grid is fine enough, that point will be in the neighborhood of the global minimum. However, users often employ some other optimization program to "polish" the gridpoint values, i.e., to seek a more precise (local) minimum near `brute's` best gridpoint. The `brute` function's `finish` option provides a convenient way to do that. Any polishing program used must take `brute's` output as its initial guess as a positional argument, and take `brute's` input values for `args` as keyword arguments, otherwise an error will be raised. It may additionally take `full_output` and/or `disp` as keyword arguments. `brute` assumes that the `finish` function returns either an `OptimizeResult` object or a tuple in the form: ``(xmin, Jmin, ... , statuscode)``, where ``xmin`` is the minimizing value of the argument, ``Jmin`` is the minimum value of the objective function, "..." may be some other returned values (which are not used by `brute`), and ``statuscode`` is the status code of the `finish` program. Note that when `finish` is not None, the values returned are those of the `finish` program, *not* the gridpoint ones. Consequently, while `brute` confines its search to the input grid points, the `finish` program's results usually will not coincide with any gridpoint, and may fall outside the grid's boundary. Thus, if a minimum only needs to be found over the provided grid points, make sure to pass in ``finish=None``. *Note 2*: The grid of points is a `numpy.mgrid` object. For `brute` the `ranges` and `Ns` inputs have the following effect. Each component of the `ranges` tuple can be either a slice object or a two-tuple giving a range of values, such as (0, 5). If the component is a slice object, `brute` uses it directly. If the component is a two-tuple range, `brute` internally converts it to a slice object that interpolates `Ns` points from its low-value to its high-value, inclusive. Examples -------- We illustrate the use of `brute` to seek the global minimum of a function of two variables that is given as the sum of a positive-definite quadratic and two deep "Gaussian-shaped" craters. Specifically, define the objective function `f` as the sum of three other functions, ``f = f1 + f2 + f3``. We suppose each of these has a signature ``(z, *params)``, where ``z = (x, y)``, and ``params`` and the functions are as defined below. >>> import numpy as np >>> params = (2, 3, 7, 8, 9, 10, 44, -1, 2, 26, 1, -2, 0.5) >>> def f1(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (a * x**2 + b * x * y + c * y**2 + d*x + e*y + f) >>> def f2(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (-g*np.exp(-((x-h)**2 + (y-i)**2) / scale)) >>> def f3(z, *params): ... x, y = z ... a, b, c, d, e, f, g, h, i, j, k, l, scale = params ... return (-j*np.exp(-((x-k)**2 + (y-l)**2) / scale)) >>> def f(z, *params): ... return f1(z, *params) + f2(z, *params) + f3(z, *params) Thus, the objective function may have local minima near the minimum of each of the three functions of which it is composed. To use `fmin` to polish its gridpoint result, we may then continue as follows: >>> rranges = (slice(-4, 4, 0.25), slice(-4, 4, 0.25)) >>> from scipy import optimize >>> resbrute = optimize.brute(f, rranges, args=params, full_output=True, ... finish=optimize.fmin) >>> resbrute[0] # global minimum array([-1.05665192, 1.80834843]) >>> resbrute[1] # function value at global minimum -3.4085818767 Note that if `finish` had been set to None, we would have gotten the gridpoint [-1.0 1.75] where the rounded function value is -2.892. (z5Brute Force not possible with more than 40 variables.rr!rN)poolrrrrrrFzhEither final optimization did not succeed or `finish` does not return `statuscode` as its last argument.rrz)#rrur rrslicerLcomplexrCmgridrreshapeprodTiterable_Brute_Wrapperr'rrrrrr}rr_getfullargspecrFdictrr=r:r0rrr)rrangesrFNsrfinishrworkersr!lranger#grid inpt_shape wrapped_funcmapperJoutNshapeindxNindxrthisNJmin finish_args finish_kwargsr[r0s r?rr3s(Z F A2vv.// / &\\F 1XX**&)U++ *6!9~~!!!&),, ~=q vay)F1I Q 8F DJ Az$A 1220G0G HIIK ;t  w!$--L  ! ! !2VxVVL$778899 FF7D:d##DD!ee:dJqrrN33D:dfj11D2222222222222224[[F $**,,R ( ( (D HQ  E 8Au  D 1q5"b ! !q &)#au} 1XX((q'%,,'Q e D QAwAw<%f--2  K ' '+,M- ( [ $(M& ! ! + % %)/~M) $fT4< * * #5D7DkGGq6Dq6D"glG < < S,<<<<T4%% sBG((G,/G,ceZdZdZdZdZdS)rnzU Object to wrap user cost function for optimize.brute, allowing picklability c*||_|gn||_dSr9)rrF)r>rrFs r?r@z_Brute_Wrapper.__init__=s,BBD rAcp|jtj|g|jRSr9)rrCrrrF)r>r=s r?rKz_Brute_Wrapper.__call__As2tvbjmm++--: ::::rAN)rNrOrPrQr@rKrRrAr?rnrn8s<111;;;;;rArnTc :ddl}ddddddd }|gd }|td d |gd|tdd |gd|tdd |gd|tdd d|}n|}||vrt d||{g}||D]Z\}}|d|zddt|zzdzg|t||d [d|}n|}t||}||vrt d|||}| d} d| dd} t| ttj | | d} | j} | (|| }nd}|rt#|dS|S)a Show documentation for additional options of optimization solvers. These are method-specific options that can be supplied through the ``options`` dict. Parameters ---------- solver : str Type of optimization solver. One of 'minimize', 'minimize_scalar', 'root', 'root_scalar', 'linprog', or 'quadratic_assignment'. method : str, optional If not given, shows all methods of the specified solver. Otherwise, show only the options for the specified method. Valid values corresponds to methods' names of respective solver (e.g., 'BFGS' for 'minimize'). disp : bool, optional Whether to print the result rather than returning it. Returns ------- text Either None (for disp=True) or the text string (disp=False) Notes ----- The solver-specific methods are: `scipy.optimize.minimize` - :ref:`Nelder-Mead ` - :ref:`Powell ` - :ref:`CG ` - :ref:`BFGS ` - :ref:`Newton-CG ` - :ref:`L-BFGS-B ` - :ref:`TNC ` - :ref:`COBYLA ` - :ref:`COBYQA ` - :ref:`SLSQP ` - :ref:`dogleg ` - :ref:`trust-ncg ` `scipy.optimize.root` - :ref:`hybr ` - :ref:`lm ` - :ref:`broyden1 ` - :ref:`broyden2 ` - :ref:`anderson ` - :ref:`linearmixing ` - :ref:`diagbroyden ` - :ref:`excitingmixing ` - :ref:`krylov ` - :ref:`df-sane ` `scipy.optimize.minimize_scalar` - :ref:`brent ` - :ref:`golden ` - :ref:`bounded ` `scipy.optimize.root_scalar` - :ref:`bisect ` - :ref:`brentq ` - :ref:`brenth ` - :ref:`ridder ` - :ref:`toms748 ` - :ref:`newton ` - :ref:`secant ` - :ref:`halley ` `scipy.optimize.linprog` - :ref:`simplex ` - :ref:`interior-point ` - :ref:`revised simplex ` - :ref:`highs ` - :ref:`highs-ds ` - :ref:`highs-ipm ` `scipy.optimize.quadratic_assignment` - :ref:`faq ` - :ref:`2opt ` Examples -------- We can print documentations of a solver in stdout: >>> from scipy.optimize import show_options >>> show_options(solver="minimize") ... Specifying a method is possible: >>> show_options(solver="minimize", method="Nelder-Mead") ... We can also get the documentations as a string: >>> show_options(solver="minimize", method="Nelder-Mead", disp=False) Minimization of scalar function of one or more variables using the ... rN))bfgsz'scipy.optimize._optimize._minimize_bfgs)cgz%scipy.optimize._optimize._minimize_cg)rVz*scipy.optimize._cobyla_py._minimize_cobyla)rWz*scipy.optimize._cobyqa_py._minimize_cobyqa)doglegz3scipy.optimize._trustregion_dogleg._minimize_dogleg)zl-bfgs-bz*scipy.optimize._lbfgsb_py._minimize_lbfgsb)z nelder-meadz-scipy.optimize._optimize._minimize_neldermead)z newton-cgz+scipy.optimize._optimize._minimize_newtoncg)powellz)scipy.optimize._optimize._minimize_powell)rUz(scipy.optimize._slsqp_py._minimize_slsqp)rTz!scipy.optimize._tnc._minimize_tnc)z trust-ncgz3scipy.optimize._trustregion_ncg._minimize_trust_ncg)r^z?scipy.optimize._trustregion_constr._minimize_trustregion_constr)z trust-exactz=scipy.optimize._trustregion_exact._minimize_trustregion_exact)z trust-krylovz9scipy.optimize._trustregion_krylov._minimize_trust_krylov) )hybrz%scipy.optimize._minpack_py._root_hybr)lmz"scipy.optimize._root._root_leastsq)broyden1z'scipy.optimize._root._root_broyden1_doc)broyden2z'scipy.optimize._root._root_broyden2_doc)andersonz'scipy.optimize._root._root_anderson_doc) diagbroydenz*scipy.optimize._root._root_diagbroyden_doc)excitingmixingz-scipy.optimize._root._root_excitingmixing_doc) linearmixingz+scipy.optimize._root._root_linearmixing_doc)krylovz%scipy.optimize._root._root_krylov_doc)zdf-sanez&scipy.optimize._spectral._root_df_sane))bisectz3scipy.optimize._root_scalar._root_scalar_bisect_doc)brentqz3scipy.optimize._root_scalar._root_scalar_brentq_doc)brenthz3scipy.optimize._root_scalar._root_scalar_brenth_doc)ridderz3scipy.optimize._root_scalar._root_scalar_ridder_doc)toms748z4scipy.optimize._root_scalar._root_scalar_toms748_doc)secantz3scipy.optimize._root_scalar._root_scalar_secant_doc)newtonz3scipy.optimize._root_scalar._root_scalar_newton_doc)halleyz3scipy.optimize._root_scalar._root_scalar_halley_doc))simplexz,scipy.optimize._linprog._linprog_simplex_doc)zinterior-pointz'scipy.optimize._linprog._linprog_ip_doc)zrevised simplexz'scipy.optimize._linprog._linprog_rs_doc)z highs-ipmz.scipy.optimize._linprog._linprog_highs_ipm_doc)zhighs-dsz-scipy.optimize._linprog._linprog_highs_ds_doc)highsz*scipy.optimize._linprog._linprog_highs_doc))faqz-scipy.optimize._qap._quadratic_assignment_faq)2optz.scipy.optimize._qap._quadratic_assignment_2opt))rz/scipy.optimize._optimize._minimize_scalar_brent)boundedz1scipy.optimize._optimize._minimize_scalar_bounded)r z0scipy.optimize._optimize._minimize_scalar_golden)minimizeroot root_scalarlinprogquadratic_assignmentminimize_scalar)z ======== z minimize z ======== rF)r)z =============== zminimize_scalar z=============== r)z ==== zroot z==== r)z ======= zlinprog z======= rrzUnknown solver z rc=zUnknown method .r)textwraprrextendr|lowerrurrpsplit __import__getattrsysmodulesrQdedentstripr) r2rjrr doc_routinestextnamer^methodspartsmod_nameobjdocs r?rrFsVOOO ,     !  s>>L@~??? L%888999 *** + + + L!2???@@@     Le444555 """ # # # L777888wwt}}  % %9v99:: : >D'/ D Da Vd]D3s4yy=,@6,IJKKK LEBBBCCCC774==DD\\^^F</00GW$$ !=6!=!=>>>6?DJJsOOExxcrc ++H x #+h/r;;C+Cs++1133  d  rAr9)r)NrRNNNN) rRrrNNrr!rNN) rRNNNFFNrrFN)rRrmrrr!)rRrmrr)rRNrrr)NrRrrr)r{rrRr!r")rCNNN) rRNNrrNNFNF)NNT)e__all__ __docformat__rrrrenumpyrrrrrrrC scipy.linalgrrrscipy.sparse.linalgr _linesearchr"r#rr$_numdiffr%scipy._lib._utilr&ror'r(r)r*r+(scipy.optimize._differentiable_functionsr,r-scipy._lib._array_apir. scipy._libr/rrr7rlr UserWarningrrwrrrrrq_epsilonrrrrr r r rrr$rrrrrrrWrYrlrrrvrrrrrrrrrrr rr r%r6rBrOrrVrrrnrrRrAr?rs6(    &  ::::::::::::::::;;;;;;;;;;......------------('''''FFFFFFMMMMMMMMMMMMMMOOOOOOOO111111------F/*33%. / /60* * * * * [* * * Z     k    L L LWWW... 4# $ $;;;;AE@D"&[[[[|      0 0 0 f'''T(4(4(4V***Z$$$0        $$$6HLIMzzzzz6:9>;?HLLLLL^"*N/N/N/N/bF+++.6TZZZZ,+Zz     |   ***Z!r26!!Q4CMMMM`"$26x%dt! nnnnbJJJJRdd!dsttttn "tdXtDSWWWWt%)2tt1tuuuup&(TD $4Xt!ey!y!y!y!x8;"#[[[[|13;<SSSSlrrrrrrrrjMMMM`