^MhddlZddlZddlmZddlmZmZddlm Z m Z ddl m Z m Z mZmZmZddlmZddlmZejejZejejZGd d ZdS) N) lsq_linear)Models Quadratic)Options Constants)cauchy_geometryspider_geometrynormal_byrd_omojokuntangential_byrd_omojokun$constrained_tangential_byrd_omojokun)qr_tangential_byrd_omojokun)get_arrays_tolceZdZdZdZedZedZedZedZ edZ edZ e j d Z ed Z e j d Z ed Zed ZedZedZedZedZedZdZdZdZdZdZdZdZdZd+dZdZdZdZ d Z!d!Z"d"Z#d#Z$d$Z%d,d%Z&d&Z'd'Z(d(Z)d)Z*d*Z+dS)- TrustRegionz! Trust-region framework. cd|_||_t|j||j|_||_d|_|tj |j |_ tj |j |_ tj |j|_tj |j|_||j|t(j|_|j|_dS)a  Initialize the trust-region framework. Parameters ---------- pb : `cobyqa.problem.Problem` Problem to solve. options : dict Options of the solver. constants : dict Constants of the solver. Raises ------ `cobyqa.utils.MaxEvalError` If the maximum number of evaluations is reached. `cobyqa.utils.TargetSuccess` If a nearly feasible point has been found with an objective function value below the target. `cobyqa.utils.FeasibleSuccess` If a feasible point has been found for a feasibility problem. `numpy.linalg.LinAlgError` If the initial interpolation system is ill-defined. rN)_penalty_pbrpenalty_models _constants _best_indexset_best_indexnpzeros m_linear_ub _lm_linear_ub m_linear_eq _lm_linear_eqm_nonlinear_ub_lm_nonlinear_ubm_nonlinear_eq_lm_nonlinear_eqset_multipliersx_bestrRHOBEG _resolution resolution_radius)selfpboptions constantss [/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/scipy/_lib/cobyqa/framework.py__init__zTrustRegion.__init__s4 dh>> #  Xd&677Xd&677 ")< = = ")< = = T[)))#7>2 c|jjS)zt Number of variables. Returns ------- int Number of variables. )rnr+s r/r3z TrustRegion.nLsxzr1c|jjS)z Number of linear inequality constraints. Returns ------- int Number of linear inequality constraints. )rrr4s r/rzTrustRegion.m_linear_ubXx##r1c|jjS)z Number of linear equality constraints. Returns ------- int Number of linear equality constraints. )rrr4s r/rzTrustRegion.m_linear_eqdr6r1c|jjS)z Number of nonlinear inequality constraints. Returns ------- int Number of nonlinear inequality constraints. )rr!r4s r/r!zTrustRegion.m_nonlinear_ubpx&&r1c|jjS)z Number of nonlinear equality constraints. Returns ------- int Number of nonlinear equality constraints. )rr#r4s r/r#zTrustRegion.m_nonlinear_eq|r9r1c|jS)zv Trust-region radius. Returns ------- float Trust-region radius. )r*r4s r/radiuszTrustRegion.radius |r1c||_|j|jtj|jzkr|j|_dSdS)z Set the trust-region radius. Parameters ---------- radius : float New trust-region radius. N)r*r<rrDECREASE_RADIUS_THRESHOLDr))r+r<s r/r<zTrustRegion.radiussJ KyBCo   ?DLLL   r1c|jS)z Resolution of the trust-region framework. The resolution is a lower bound on the trust-region radius. Returns ------- float Resolution of the trust-region framework. r(r4s r/r)zTrustRegion.resolutions r1c||_dS)z Set the resolution of the trust-region framework. Parameters ---------- resolution : float New resolution of the trust-region framework. NrA)r+r)s r/r)zTrustRegion.resolutions&r1c|jS)zr Penalty parameter. Returns ------- float Penalty parameter. )rr4s r/rzTrustRegion.penaltys }r1c|jS)z Models of the objective function and constraints. Returns ------- `cobyqa.models.Models` Models of the objective function and constraints. )rr4s r/modelszTrustRegion.modelsr=r1c|jS)z Index of the best interpolation point. Returns ------- int Index of the best interpolation point. )rr4s r/ best_indexzTrustRegion.best_indexs r1cJ|jj|jS)z Best interpolation point. Its value is interpreted as relative to the origin, not the base point. Returns ------- `numpy.ndarray` Best interpolation point. )rE interpolationpointrGr4s r/r&zTrustRegion.x_bests{(..t???r1c0|jj|jS)z Value of the objective function at `x_best`. Returns ------- float Value of the objective function at `x_best`. )rEfun_valrGr4s r/fun_bestzTrustRegion.fun_bests{"4?33r1c8|jj|jddfS)z Values of the nonlinear inequality constraints at `x_best`. Returns ------- `numpy.ndarray`, shape (m_nonlinear_ub,) Values of the nonlinear inequality constraints at `x_best`. N)rEcub_valrGr4s r/cub_bestzTrustRegion.cub_best{"4?AAA#566r1c8|jj|jddfS)z Values of the nonlinear equality constraints at `x_best`. Returns ------- `numpy.ndarray`, shape (m_nonlinear_eq,) Values of the nonlinear equality constraints at `x_best`. N)rEceq_valrGr4s r/ceq_bestzTrustRegion.ceq_best rQr1c~|j||j|jjj|z|jjjz zz|j|jjj|z|jjj z zz|j |j |zz|j |j |zzS)a/ Evaluate the Lagrangian model at a given point. Parameters ---------- x : `numpy.ndarray`, shape (n,) Point at which the Lagrangian model is evaluated. Returns ------- float Value of the Lagrangian model at `x`. )rEfunrrlineara_ubb_ubr a_eqb_eqr"cubr$ceqr+xs r/ lag_modelzTrustRegion.lag_models KOOA   x#a'$(/*>>@ @ x#a'$(/*>>@ @ #dkooa&8&88  9 #dkooa&8&88  9 r1c*|j||j|jjjzz|j|jjjzz|j|j |zz|j |j |zzS)ah Evaluate the gradient of the Lagrangian model at a given point. Parameters ---------- x : `numpy.ndarray`, shape (n,) Point at which the gradient of the Lagrangian model is evaluated. Returns ------- `numpy.ndarray`, shape (n,) Gradient of the Lagrangian model at `x`. ) rEfun_gradrrrWrXr rZr"cub_gradr$ceq_gradr^s r/lag_model_gradzTrustRegion.lag_model_grad.s K  # # 48?#77 8 48?#77 8#dk&:&:1&=&== >#dk&:&:1&=&==  > r1c|j}|jdkr$||j|jzz }|jdkr$||j|jzz }|S)z Evaluate the Hessian matrix of the Lagrangian model at a given point. Returns ------- `numpy.ndarray`, shape (n, n) Hessian matrix of the Lagrangian model at `x`. r)rEfun_hessr!r"cub_hessr#r$ceq_hess)r+hesss r/lag_model_hesszTrustRegion.lag_model_hessDsx{##%%   " " D)DK,@,@,B,BB BD   " " D)DK,@,@,B,BB BD r1c|j||j|j|zz|j|j|zzS)a Evaluate the right product of the Hessian matrix of the Lagrangian model with a given vector. Parameters ---------- v : `numpy.ndarray`, shape (n,) Vector with which the Hessian matrix of the Lagrangian model is multiplied from the right. Returns ------- `numpy.ndarray`, shape (n,) Right product of the Hessian matrix of the Lagrangian model with `v`. )rE fun_hess_prodr" cub_hess_prodr$ ceq_hess_prodr+vs r/lag_model_hess_prodzTrustRegion.lag_model_hess_prodTs^$ K % %a ( (#dk&?&?&B&BB C#dk&?&?&B&BB C r1c|j||j|j|zz|j|j|zzS)ar Evaluate the curvature of the Lagrangian model along a given direction. Parameters ---------- v : `numpy.ndarray`, shape (n,) Direction along which the curvature of the Lagrangian model is evaluated. Returns ------- float Curvature of the Lagrangian model along `v`. )rEfun_curvr"cub_curvr$ceq_curvrps r/lag_model_curvzTrustRegion.lag_model_curvks\ K  # ##dk&:&:1&=&== >#dk&:&:1&=&== > r1cx||j|jd||zzzS)a} Evaluate the objective function of the SQP subproblem. Parameters ---------- step : `numpy.ndarray`, shape (n,) Step along which the objective function of the SQP subproblem is evaluated. Returns ------- float Value of the objective function of the SQP subproblem along `step`. g?)rErbr&rrr+steps r/sqp_funzTrustRegion.sqp_funsA K  - -D,,T222 3  r1c|j|j|j|j|zzS)a Evaluate the linearization of the nonlinear inequality constraints. Parameters ---------- step : `numpy.ndarray`, shape (n,) Step along which the linearization of the nonlinear inequality constraints is evaluated. Returns ------- `numpy.ndarray`, shape (m_nonlinear_ub,) Value of the linearization of the nonlinear inequality constraints along `step`. )rEr\r&rcrys r/sqp_cubzTrustRegion.sqp_cub;" KOODK ( (k""4;//$6 7 r1c|j|j|j|j|zzS)a Evaluate the linearization of the nonlinear equality constraints. Parameters ---------- step : `numpy.ndarray`, shape (n,) Step along which the linearization of the nonlinear equality constraints is evaluated. Returns ------- `numpy.ndarray`, shape (m_nonlinear_ub,) Value of the linearization of the nonlinear equality constraints along `step`. )rEr]r&rdrys r/sqp_ceqzTrustRegion.sqp_ceqr~r1Nc ||||||j\}}}|}|jdkr[|j|||}t j|r*||jtj|zz }|S)av Evaluate the merit function at a given point. Parameters ---------- x : `numpy.ndarray`, shape (n,) Point at which the merit function is evaluated. fun_val : float, optional Value of the objective function at `x`. If not provided, the objective function is evaluated at `x`. cub_val : `numpy.ndarray`, shape (m_nonlinear_ub,), optional Values of the nonlinear inequality constraints. If not provided, the nonlinear inequality constraints are evaluated at `x`. ceq_val : `numpy.ndarray`, shape (m_nonlinear_eq,), optional Values of the nonlinear equality constraints. If not provided, the nonlinear equality constraints are evaluated at `x`. Returns ------- float Value of the merit function at `x`. Nr)rOrS)rrr violationr count_nonzerolinalgnorm)r+r_rLrOrSm_valc_vals r/meritzTrustRegion.merits. ?go(,DL(A(A %GWg =3  H&&q'7&KKE&& ?)>)>>> r1cVtj|jjjg|j|gg}tj|jjj|jjj|zz |j| g}tj|jjj g|j |gg}tj|jjj |jjj |zz |j | g}||||fS)a; Get the linearizations of the constraints at a given point. Parameters ---------- x : `numpy.ndarray`, shape (n,) Point at which the linearizations of the constraints are evaluated. Returns ------- `numpy.ndarray`, shape (m_linear_ub + m_nonlinear_ub, n) Left-hand side matrix of the linearized inequality constraints. `numpy.ndarray`, shape (m_linear_ub + m_nonlinear_ub,) Right-hand side vector of the linearized inequality constraints. `numpy.ndarray`, shape (m_linear_eq + m_nonlinear_eq, n) Left-hand side matrix of the linearized equality constraints. `numpy.ndarray`, shape (m_linear_eq + m_nonlinear_eq,) Right-hand side vector of the linearized equality constraints. ) rblockrrWrXrErcrYr\rZrdr[r])r+r_aubbubaeqbeqs r/get_constraint_linearizationsz)TrustRegion.get_constraint_linearizationss(h%&%%a(()    h$tx';a'??###    h%&%%a(()    h$tx';a'??###    Cc!!r1c ||j\}}}}|jjj|jz }|jjj|jz }|jtj|j z}t||||||||tj fi|j} |tj rt||} tj| | z|kstj|| | z krt!jdt$dtj| d|zkrt!jdt$dtj|j dz| | zz }|| z}|| z}tj||| zz d}|j|j|| z} |jjdvr-t7| |j||||tj fi|j} n%t9| |j|||||||df i|j} |tj rt||} tj| | z|kstj|| | z krt!jd t$dtj| | zdtjdz|j zkrt!jd t$d| | fS) aK Get the trust-region step. The trust-region step is computed by solving the derivative-free trust-region SQP subproblem using a Byrd-Omojokun composite-step approach. For more details, see Section 5.2.3 of [1]_. Parameters ---------- options : dict Options of the solver. Returns ------- `numpy.ndarray`, shape (n,) Normal step. `numpy.ndarray`, shape (n,) Tangential step. References ---------- .. [1] T. M. Ragonneau. *Model-Based Derivative-Free Optimization Methods and Software*. PhD thesis, Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, 2022. URL: https://theses.lib.polyu.edu.hk/handle/200/12294. z6the normal step does not respect the bound constraint.皙?z=the normal step does not respect the trust-region constraint.@r) unconstrainedzbound-constraineddebugz;The tangential step does not respect the bound constraints.zjzkrj|z |z}  | jj dkr| } tj!|| z | |z g}| | z| z }|| z|z }tEd ||tj|fD}tj"tj| |d }tj"tj| || }tj"tj| |ddf| z| }tGd |zd tj| z}||kr\jj | z|} t-| d t-|zkrtj$| ||}|tjrt3||}tj%||z|kstj%|||z krtMj'dtPdtj|djzkrtMj'dtPd|S)a Get the geometry-improving step. Three different geometry-improving steps are computed and the best one is returned. For more details, see Section 5.2.7 of [1]_. Parameters ---------- k_new : int Index of the interpolation point to be modified. options : dict Options of the solver. Returns ------- `numpy.ndarray`, shape (n,) Geometry-improving step. Raises ------ `numpy.linalg.LinAlgError` If the computation of a determinant fails. References ---------- .. [1] T. M. Ragonneau. *Model-Based Derivative-Free Optimization Methods and Software*. PhD thesis, Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, 2022. URL: https://theses.lib.polyu.edu.hk/handle/200/12294. z8The index `k_new` must be different from the best index.rrcD|jjSNcurvrErIrqlagr+s r/z/TrustRegion.get_geometry_step..chhq$+";<<r1NrcD|jjSrrrs r/rz/TrustRegion.get_geometry_step..rr1)zlinearly constrainedznonlinearly constrainedc3BK|]}tj|dVdS)rinitialN)rmax).0arrays r/ z0TrustRegion.get_geometry_step..sE  F5#...      r1r$@g{Gz?g?z9The geometry step does not respect the bound constraints.rrz?The geometry step does not respect the trust-region constraint.))rrrGrsqueezeeyerEnptrrIgradr&rrrrr r< determinantsxptnewaxisr absrrrrTrrr3TINYrrrmincliprrrr)r+k_newr- coord_vecg_lagrrrzsigmarstep_alt sigma_altrrrrtol_bdtol_ubfree_xlfree_xufree_ubn_actq g_lag_projnorm_g_lag_projcbdr\r] maxcv_valrrs` @r/get_geometry_stepzTrustRegion.get_geometry_steps> 7= ! J(((I)((Jrva%@@AA  K %  GM "   dk&?@@X_ $+ - X_ $+ -   < < < < <   K GM "   ((t);UCC K % )k'+AAAt ,JK L (+111t.B+B'CAAA4?# #$"   < < < < < 122J   K GM "   K,,T[8-CUKK y>>CJJ & &DE 8=   224;?? Cc3#B++F#C((FVGmGFlGVmG3 HE1111eff9111eff9%)?@J innZ88O5%%%%48:%%%%%/D4;>Hns*Hns*  "%sBF3KK!8    fRVHgX$677EEEfRVHgX$677EEEfRVC!!! $4x$?@@#NNN$*dRY^^H-E-E&EFF## $ 8 8 h.!!I9~~s5zz)999!wxR88 7= !  R((CvdSj2o&& "&dSj*A*A  #"  y~~d##cDK&777 /"   r1c ||j\}}}}|jjj|jz }|jjj|jz }t j|} t||||||| |tj fi|j } |tj rt||} t j| | z|kst j|| | z krtjdt"dt j| d| zkrtjdt"d| S)aQ Get the second-order correction step. Parameters ---------- step : `numpy.ndarray`, shape (n,) Trust-region step. options : dict Options of the solver. Returns ------- `numpy.ndarray`, shape (n,) Second-order correction step. zHThe second-order correction step does not respect the bound constraints.rrzNThe second-order correction step does not respect the trust-region constraint.)rr&rrrrrrrr rrrrrrrr) r+rzr-rrrrrrr<soc_steprs r/ get_second_order_correction_stepz,TrustRegion.get_second_order_correction_stepsS""?? LLS#s X_ $+ - X_ $+ -%%'        GM "   o    7= !  R((Cvhnr)** bfR(S.5H.I.I  5"  y~~h''#,66 ;"  r1c||j|j|j|j}||j|z|||}||jd|j|j|j|j}||j|z||| || |}t||z tt||z zkr||z t||z z SdS)a: Get the reduction ratio. Parameters ---------- step : `numpy.ndarray`, shape (n,) Trust-region step. fun_val : float Objective function value at the trial point. cub_val : `numpy.ndarray`, shape (m_nonlinear_ub,) Nonlinear inequality constraint values at the trial point. ceq_val : `numpy.ndarray`, shape (m_nonlinear_eq,) Nonlinear equality constraint values at the trial point. Returns ------- float Reduction ratio. r) rr&rMrPrTrEr\r]r{r}rrr) r+rzrLrOrS merit_old merit_newmerit_model_oldmerit_model_news r/get_reduction_ratiozTrustRegion.get_reduction_ratioIs>(JJ K M M M   JJt{T17GWMM ** K  KOODK ( ( KOODK ( (    ** K$  LL   LL   LL      0 1 1D3  !< < 5    )S/1.. 4r1c ||j\}}}}ttjtjtjd| |gtjtjtjd||z|z ||z|z gz d}||}tjtj|j |j |j |j g}t|tt|zkrt|||z }|j} |j|jt$j|zkrAt|jt$j|zd|_|| |jkS)z Increase the penalty parameter. Parameters ---------- step : `numpy.ndarray`, shape (n,) Trust-region step. r?)rr&rrrrrrr{rr r"r$rrrGrrrPENALTY_INCREASE_THRESHOLDPENALTY_INCREASE_FACTORr) r+rzrrrr viol_diffsqp_val thresholdbest_index_saves r/increase_penaltyzTrustRegion.increase_penaltyys"?? LLS#s INN 3--  inn 3d S(899d S(  #  &,,t$$INN H&&))       y>>D3w<</ / /Iw':;;I/ MyCD    ABYNDM    ! ! !$/11r1ct|j||_|dS)z1 Decrease the penalty parameter. N)rr_get_low_penaltyrr4s r/decrease_penaltyzTrustRegion.decrease_penaltys;DM4+@+@+B+BCC  r1c |j}||j|jj||jj|ddf|jj|ddf}|j|j|jj|ddf|jj|ddf}dtzt|jj |jj ztt|dz}t|jj D]}||jkr|jj|}|||jj||jj|ddf|jj|ddf}|j||jj|ddf|jj|ddf}||ks|||zkr ||kr|}|}|}||_dS)z2 Set the index of the best point. Nrr)rGrr&rErLrOrSrmaxcvEPSrr3rrrangerIrJr) r+rGm_bestr_bestrkx_valrr_vals r/rzTrustRegion.set_best_indexs_  K K  + K  AAA . K  AAA .     K K  AAA . K  AAA .    $+-11 2#f++s## $ t{'' # #ADO## 177:: K'*K'111-K'111-  K'111-K'111- 6>>efsl&:&:uv~~!"J"F"F%r1c&tj|jjj|jjjdd|jtjfz dzd}|d}|}nr|j|}tjd|t|j tj |j z|jdzz dz}d||j<tj|tj|z}|tj||fS)a Get the index of the interpolation point to remove. If `x_new` is not provided, the index returned should be used during the geometry-improvement phase. Otherwise, the index returned is the best index for included `x_new` in the interpolation set. Parameters ---------- x_new : `numpy.ndarray`, shape (n,), optional New point to be included in the interpolation set. Returns ------- int Index of the interpolation point to remove. float Distance between `x_best` and the removed point. Raises ------ `numpy.linalg.LinAlgError` If the computation of a determinant fails. Nrraxisrg@r)rsumrErIrrGrrrrrrLOW_RADIUS_FACTORr<r)argmaxrr)r+x_newdist_sqrweightsk_maxs r/get_index_to_removezTrustRegion.get_index_to_removes 2& )-++/4?BJ0NOP        =EGGK,,U33E  (CD+&      (,GDO $ 'BF5MM122bggen----r1cPtj|}||jtjkr'|xj|jtjzc_dS||jtjkr4t|jtj|jz||_dSt|jtj |jzt|jtj|jz|jtj |z|_dS)z Update the trust-region radius. Parameters ---------- step : `numpy.ndarray`, shape (n,) Trust-region step. ratio : float Reduction ratio. N) rrrrr LOW_RATIOr<DECREASE_RADIUS_FACTOR HIGH_RATIOrrINCREASE_RADIUS_FACTORINCREASE_RADIUS_THRESHOLD)r+rzratios_norms r/ update_radiuszTrustRegion.update_radiuss%% DOI$78 8 8 KK4?9+KL LKKKK doi&:; ; ; @A+DKKK  @A+OI$DEk"OI$GH  DKKKr1c|jtj|tjz|jkr&|xj|jtjzc_n||jtj|tjz|jkr2tj |j|tjz|_n|tj|_t|jtj |j z|j|_ dS)z Enhance the resolution of the trust-region framework. Parameters ---------- options : dict Options of the solver. N) rrLARGE_RESOLUTION_THRESHOLDrRHOENDr)DECREASE_RESOLUTION_FACTORMODERATE_RESOLUTION_THRESHOLDrrrrr*r+r-s r/enhance_resolutionzTrustRegion.enhance_resolution9s OI@ Agn% &o   OOt4  OOO OIC Dgn% &o  !gdo(/(?'@AADOO&gn5DO OI< = L O   r1cj|jtj|j|dS)z Shift the base point to `x_best`. Parameters ---------- options : dict Options of the solver. N)rE shift_x_basercopyr&rs r/rzTrustRegion.shift_x_baseZs.   !5!5w?????r1c D|jjj|z|jjjk}|jdk}|jjj|k}|jjj|k}tj |}tj |}tj |}tj |} ||z|j z|j zdkrtj |jj } tj| |ddf | |ddf|jjj|ddf|j|||jjj|j|f} |j|} tj| jdtj } d| d|| z|z|z<t/| j| | tjfd}|j|| z|| z|z|j|<d|j|<|j|| z|z|| z|z|z|j|<d|j|<|j|| z|z|z|| z|z|z|j z|jdd<|j|| z|z|z|j zd|jdd<dSdS)aI Set the Lagrange multipliers. This method computes and set the Lagrange multipliers of the linear and nonlinear constraints to be the QP multipliers. Parameters ---------- x : `numpy.ndarray`, shape (n,) Point at which the Lagrange multipliers are computed. rrNbvls)rmethod)rrWrXrYrPrrrrrrr#rr3r_rErcrZrdrbfullshapeinfrrr_rr"r r$)r+r_incl_linear_ubincl_nonlinear_ubincl_xlincl_xurr!m_xlm_xuidentityc_jacrxl_lmress r/r%zTrustRegion.set_multiplierses-1TX_5II MS0(/$)(/$)&~66 )*;<<(((( . (4+; ;% &() * *vdhj))HE'111*%%!!!$$^QQQ%67 $$Q(9::$ $$Q'' )E[))!,,FGEKNRVG44EBEE>D4K+->> ?rv C25t D4K+552D ~ .36D  /7:u"!!!8D !"3 49rFs*%%%%%%%%%%%%%%((((((((:99999!!!!!!rxbhuooAAAAAAAAAAr1