M/PhC dZddlZddlZddlZddlmZddlm Z ddl m cm Z ddlmZddlmZmZmZddlmZmZddlmZdd lmZmZddlmcm cmZdd l m!Z!m"Z"m#Z#m$Z$dd l%m&Z&m'Z'm(Z(m)Z)dd l*m+Z+m,Z,m-Z-e#e"e!e$d Z.e(e'e&e)d Z/dZ0dZ1dZ2dZ3GddZ4Gddej5Z6GddZ7Gdde7Z8Gddej9Z:Gdde j;Z<e j=eC 26!i!&<&<&<"<==qwGGI ct|jdkrdzz Sjdkrtjdzz }n$jdkrfdt jdD}jdjdjdf}tj||d}nfdt jdD}jdjdjdjdf}tj||d}t jdD]A}t |D]/}|df |dfz|||df<|||df|||df<0B|S) rrr rclg|]0}tjdd|fdd|fdzz 1SNr r%diag).0tr2s r4 z&_partials_logistic..ZsT222GC1IAAAqD 1 455222r6)rr rc`g|])fdtjdD*S)c pg|]2}tjdd|fdd|fdzz 3Sr9r:)r<r=jr2s r4r>z1_partials_logistic...`sX333WSAq\C1aL!O;<<333r6r )ranger*)r<rAr2s @r4r>z&_partials_logistic..`s^22233333#CIaL11333222r6)r r!rr.) r5r'r%r;rBr* concatenater( transpose)r0partialsr*irAr2s @r4_partials_logisticrGLs A,,C x1}}S!V| Q73a<(( Q2222"39Q<00222 ! cilCIaL8>(++33E::DDYOO2222"39Q<00222 ! cilCIaL#)A,F>(++33E::DD 39Q< 66q 6 6A#&q#v;,QV" ? ?66*,& +"+6+6+6'' ,-J ">sABBw"G - ,S!""W 5 77r6c |jd}|jd}|jdz }|j}tj|f|dzz|fz|}|jd||zkr |d|df}tjtj|d}t|||f\}}} t|} | ||||| ||dzz|| ||dzz|| ||dzz|tj |}|} td| jdz D]} tj | d } || fS) a Kim smoother in log space using Cython inner loop. Parameters ---------- regime_transition : ndarray Matrix of regime transition probabilities, shaped either (k_regimes, k_regimes, 1) or if there are time-varying transition probabilities (k_regimes, k_regimes, nobs). predicted_joint_probabilities : ndarray Array containing Pr[S_t=s_t, ..., S_{t-order}=s_{t-order} | Y_{t-1}] - the joint probability of the current and previous `order` periods being in each combination of regimes conditional on time t-1 information. Shaped (k_regimes,) * (order + 1) + (nobs,). filtered_joint_probabilities : ndarray Array containing Pr[S_t=s_t, ..., S_{t-order}=s_{t-order} | Y_{t}] - the joint probability of the current and previous `order` periods being in each combination of regimes conditional on time t information. Shaped (k_regimes,) * (order + 1) + (nobs,). Returns ------- smoothed_joint_probabilities : ndarray Array containing Pr[S_t=s_t, ..., S_{t-order}=s_{t-order} | Y_T] - the joint probability of the current and previous `order` periods being in each combination of regimes conditional on all information. Shaped (k_regimes,) * (order + 1) + (nobs,). smoothed_marginal_probabilities : ndarray Array containing Pr[S_t=s_t | Y_T] - the probability of being in each regime conditional on all information. Shaped (k_regimes, nobs). rr"r rrJ.NrIrNr#) r*r'rKr%r-rPrQrprefix_kim_smoother_log_mapr(r/rBrT) rVr^r`rYrZr[rKsmoothed_joint_probabilitiesrbrcrdsmoothed_marginal_probabilitiesrFs r4cy_kim_smoother_logrlsF-215I ' -b 1D ( - 1E ( .E$&8   "dW,E$;$;$; r"dUl22-c566k:rz*;UCCDD+8$,&''FE1 'v .DDy%!2 & . .y57/CT J J % - -i%'.BD I I % - -i%'.BD I IKKK $&6*F#G#G 'C# 15:Q> ? ?66*,& +"+6+6+6'' ()H HHr6c$eZdZdZdZdZdZdS)MarkovSwitchingParamsa= Class to hold parameters in Markov switching models Parameters ---------- k_regimes : int The number of regimes between which parameters may switch. Notes ----- The purpose is to allow selecting parameter indexes / slices based on parameter type, regime number, or both. Parameters are lexicographically ordered in the following way: 1. Named type string (e.g. "autoregressive") 2. Number (e.g. the first autoregressive parameter, then the second) 3. Regime (if applicable) Parameter blocks are set using dictionary setter notation where the key is the named type string and the value is a list of boolean values indicating whether a given parameter is switching or not. For example, consider the following code: parameters = MarkovSwitchingParams(k_regimes=2) parameters['regime_transition'] = [1,1] parameters['exog'] = [0, 1] This implies the model has 7 parameters: 4 "regime_transition"-related parameters (2 parameters that each switch according to regimes) and 3 "exog"-related parameters (1 parameter that does not switch, and one 1 that does). The order of parameters is then: 1. The first "regime_transition" parameter, regime 0 2. The first "regime_transition" parameter, regime 1 3. The second "regime_transition" parameter, regime 1 4. The second "regime_transition" parameter, regime 1 5. The first "exog" parameter 6. The second "exog" parameter, regime 0 7. The second "exog" parameter, regime 1 Retrieving indexes / slices is done through dictionary getter notation. There are three options for the dictionary key: - Regime number (zero-indexed) - Named type string (e.g. "autoregressive") - Regime number and named type string In the above example, consider the following getters: >>> parameters[0] array([0, 2, 4, 6]) >>> parameters[1] array([1, 3, 5, 6]) >>> parameters['exog'] slice(4, 7, None) >>> parameters[0, 'exog'] [4, 6] >>> parameters[1, 'exog'] [4, 7] Notice that in the last two examples, both lists of indexes include 4. That's because that is the index of the the non-switching first "exog" parameter, which should be selected regardless of the regime. In addition to the getter, the `k_parameters` attribute is an dict with the named type strings as the keys. It can be used to get the total number of parameters of each type: >>> parameters.k_parameters['regime_transition'] 4 >>> parameters.k_parameters['exog'] 3 c||_d|_i|_i|_i|_dt |jD|_dt |jD|_dt |jD|_dS)Nrcg|]}iSrqr<rFs r4r>z2MarkovSwitchingParams.__init__..s%./././B./././r6cg|]}iSrqrqrrs r4r>z2MarkovSwitchingParams.__init__..s%%/%/%/B%/%/%/r6cg|]}gSrqrqrrs r4r>z2MarkovSwitchingParams.__init__..s???AR???r6) rYk_params k_parameters switchingslices_purposerBrelative_index_regime_purposeindex_regime_purpose index_regime)selfrYs r4__init__zMarkovSwitchingParams.__init__s"  ././dn--./././*%/%/dn--%/%/%/!??t~)>)>???r6cft|}|tur |j|S|tur |j|S|t urt |dkstdt|dtur;t|dtur|j|d|dSt|dtur;t|dtur|j|d|dStdtd)Nr Invalid indexrr) typestrrxintr{tupler) IndexErrorrz)r|key_types r4 __getitem__z!MarkovSwitchingParams.__getitem__sS  C<<&s+ + c\\$S) ) e^^s88q== 111CF||s""tCF||s':':0Q8Q@@c!f$$c!f)<)<0Q8Q@@ 111_-- -r6ct|}|tur9tj|td}|j}|jtj||jdz zz|j |<|xj|j |z c_||j |<tj ||j|j |<t|jD]"}g|j||<g|j||<#d}t|jD]}||}t|jD]T}|s'|j|||+|j||||zU||sdn|jz }t|jD]}d}g} |j|D]a\} } tj| |z} | |j|| <| | ||j | z }btj| t,|j|<dSt1d)Nr)rKndminrr)rrr%r&boolrusizerTrYrvrws_rxrBryrzappenditemsr_tolistrCastyperr{r) r|rvaluerrurAoffsetrFrwindiceskvs r4 __setitem__z!MarkovSwitchingParams.__setitem__sS  C<<HU$a888E}H RVE]]dnq.@AA  c " MMT.s3 3MM"'DN3 ')uXdm-C'DD  $4>** 7 7=?215c:46)!,S11F5:&& A A!!H t~..((A$(:1=cBII"$$$$:1=cBII"QJ((((9@!!$.@4>** K K >qAGGII33DAqqF*2244A67D-a03NN1%%%d/22FF')~g'>'>'E'Ec'J'J!!$$ K K_-- -r6N)__name__ __module__ __qualname____doc__r}rrrqr6r4rnrn4sQMM\ @ @ @...*(.(.(.(.(.r6rncleZdZdZ d/fd ZedZdZd0d Zd1d Z d1d Z d1d Z d2dZ dZ dZd1dZ d3dZ d1dZedZ d4dZ d3dZd5dZd5dZd5dZd5dZd5dZ d6fd# Z d7d&Zd'Zd(Z d8d)Zed*Zed+Z d,Z!d-Z"d.Z#xZ$S)9MarkovSwitchingaU First-order k-regime Markov switching model Parameters ---------- endog : array_like The endogenous variable. k_regimes : int The number of regimes. order : int, optional The order of the model describes the dependence of the likelihood on previous regimes. This depends on the model in question and should be set appropriately by subclasses. exog_tvtp : array_like, optional Array of exogenous or lagged variables to use in calculating time-varying transition probabilities (TVTP). TVTP is only used if this variable is provided. If an intercept is desired, a column of ones must be explicitly included in this array. Notes ----- This model is new and API stability is not guaranteed, although changes will be made in a backwards compatible way if possible. References ---------- Kim, Chang-Jin, and Charles R. Nelson. 1999. "State-Space Models with Regime Switching: Classical and Gibbs-Sampling Approaches with Applications". MIT Press Books. The MIT Press. rNnonec ||_|du|_||_t|\|_|_t ||||||jj d|_ |jj dkr%|jj ddkrtd|jdkrtd|j*|jj d|j kstdt|j|_|jdz } |jr | |jz} dg| z|jd<d |_d|_dS) N)datesfreqmissingrrz%Must have univariate endogenous data.r z7Markov switching models must have at least two regimes.zxTime-varying transition probabilities exogenous array must have the same number of observations as the endogenous array.rV steady-state)rYtvtpr[rk_tvtp exog_tvtpsuperr}endogr*rZr'rOrn parameters_initialization_initial_probabilities) r|rrYr[rexogrrr k_transition __class__s r4r}zMarkovSwitching.__init__sg#T)  '39&=&=# T^  4u4    J$Q'  :?Q  4:#3A#6#:#:DEE E >A  )** *&$Q'49449:: : 0??~) 9 ( DK 'L01s\/A+, .&*###r6c|jjS)z9 (int) Number of parameters in the model )rrur|s r4ruzMarkovSwitching.k_paramss ''r6cN|jrtdd|_d|_dS)z Set initialization of regime probabilities to be steady-state values Notes ----- Only valid if there are not time-varying transition probabilities. zYCannot use steady-state initialization when the regime transition matrix is time-varying.rN)rrOrrrs r4initialize_steady_statez'MarkovSwitching.initialize_steady_state$sB 9 ONOO O .&*###r6:0yE>cd|_tj|d}|j|jfkst dtjtj|dz |kst d||_dS)zP Set initialization of regime probabilities to use known values knownrrz=Initial probabilities must be a vector of shape (k_regimes,).z-Initial probabilities vector must sum to one.N) rr%r&r*rYrOabsrTr)r| probabilitiestols r4initialize_knownz MarkovSwitching.initialize_known3s 'a888 "t~&777.// /vbf]++a/00366LMM M&3###r6cRtj|d}|jdkr|||}|jdkr|d}|jd}tjtj||z jtj |fj} tj |dddf}nI#tj j $rtd wxYw|jd kr|j}ntd tj|d }|S) z0 Retrieve initial probabilities rrrNr!rMrr"z4Steady-state probabilities could not be constructed.rz'Invalid initialization method selected.rI)r%r&rregime_transition_matrixr'r*r,eyer.oneslinalgpinv LinAlgError RuntimeErrorrrQ)r|paramsrVmArs r4rUz%MarkovSwitching.initial_probabilities@s3&***  > 1 1 ($($A$A&$I$I! %**$5f$=!!'*Arvayy#447CDFA 4 " q 1 1!!!R% 8 9( 4 4 4"$3444 4 !W , , 7MMHII I =%88 s )C $C-c "||j}t|}tj|j|j|ftjtj|j}t|jD]b}||j |df}tj |tj ||jdz |j fj j |dd|ddf<ctjtj||jdf|ddddddfj fj }tj|ddddddft!|dz |ddddddf<dtj|ddddddfdz |dddddf<|S)NrJrVrr"rr#)rr)r%r-rY promote_typesfloat64rKrBrdotr(rr.r,r/rrT)r|rrrZrrFcoeffsr2s r4_regime_transition_matrix_tvtpz.MarkovSwitching._regime_transition_matrix_tvtp\s  I9~~#%8 ^T^T 2"2:v|<<$>$>$> t~&& I IADOA/B,BCDF24& 6DN1$4dk#BCCE3G3GGH %SbS!QQQY / / eBHdDNA677,SbS!!!QQQY79:;;< .0f $SbS!!!QQQY /)Ca2H2H2H H/J/J "aaa+ /QQQ :CCC C !QQQ*('r6ctj|d}|jstj|j|jdftjtj|j}tj||j d|jdz |jf|dddddf<dtj |dddddfdz |ddddf<n| ||}|S) a Construct the left-stochastic transition matrix Notes ----- This matrix will either be shaped (k_regimes, k_regimes, 1) or if there are time-varying transition probabilities, it will be shaped (k_regimes, k_regimes, nobs). The (i,j)th element of this matrix is the probability of transitioning from regime j to regime i; thus the previous regime is represented in a column and the next regime is represented by a row. It is left-stochastic, meaning that each column sums to one (because it is certain that from one regime (j) you will transition to *some other regime*). rrrJrVNr"rr#) r%r&rr-rYrrrKr(rrTr)r|rrrs r4rz(MarkovSwitching.regime_transition_matrixxs $&***y H')x3&rz6<@@(B(B(B $35*t':;<!4>23434 $SbS!!!QY /BF3CRCAI>QGGGG %RAX . .33FIFF %('r6Fcn| |jd}|||\}}}}|dkrt||}t j|} | jdz |kr||dkr||d} | j}nI|dkr| |d} | j }n$|dkr| |d} | j }||z}t|jdz t|z D]} t j|d }n| }||||zdzS) a  In-sample prediction and out-of-sample forecasting Parameters ---------- params : ndarray Parameters at which to form predictions start : int, str, or datetime, optional Zero-indexed observation number at which to start forecasting, i.e., the first forecast is start. Can also be a date string to parse or a datetime type. Default is the the zeroth observation. end : int, str, or datetime, optional Zero-indexed observation number at which to end forecasting, i.e., the last forecast is end. Can also be a date string to parse or a datetime type. However, if the dates index does not have a fixed frequency, end must be an integer index if you want out of sample prediction. Default is the last observation in the sample. probabilities : str or array_like, optional Specifies the weighting probabilities used in constructing the prediction as a weighted average. If a string, can be 'predicted', 'filtered', or 'smoothed'. Otherwise can be an array of probabilities to use. Default is smoothed. conditional : bool or int, optional Whether or not to return predictions conditional on current or past regimes. If False, returns a single vector of weighted predictions. If True or 1, returns predictions conditional on the current regime. For larger integers, returns predictions conditional on the current regime and some number of past regimes. Returns ------- predict : ndarray Array of out of in-sample predictions and / or out-of-sample forecasts. NrrsmoothedT) return_rawfiltered predictedrNr#)_index_get_prediction_indexr+predict_conditionalr%squeezer'smoothrjfilterr`r^rBrrT) r|rstartendr conditional out_of_sampleprediction_indexpredictsqueezedresultsrFs r4rzMarkovSwitching.predictswL =KNE  & &uc 2 2 4sM#3 1  % %**622:g&& =1 { * *$ (C(C++f+>> ' D *,,++f+>> ' D +--++f+>> ' E .G7.>>?? 3 3&r222 3GuS=014455r6ct)a In-sample prediction, conditional on the current, and possibly past, regimes Parameters ---------- params : array_like Array of parameters at which to perform prediction. Returns ------- predict : array_like Array of predictions conditional on current, and possibly past, regimes r+r|rs r4rz#MarkovSwitching.predict_conditionals "!r6ct)z Compute likelihoods conditional on the current period's regime (and the last self.order periods' regimes if self.order > 0). Must be implemented in subclasses. rrs r4_conditional_loglikelihoodsz+MarkovSwitching._conditional_loglikelihoodss "!r6c|||}|||}||}|||ft||||jzSN)rrUrrgr[)r|rrVrUrWs r4_filterzMarkovSwitching._filters  $ $ = =f E E  $ : : %!'!'&*%E%Ef%M%M"#$9+-&)+<. <<< =r6TcLtj|d}|s||}|j|j_gd}t |t ditt|| |} | || |||||S)a Apply the Hamilton filter Parameters ---------- params : array_like Array of parameters at which to perform filtering. transformed : bool, optional Whether or not `params` is already transformed. Default is True. cov_type : str, optional See `fit` for a description of covariance matrix types for results object. cov_kwds : dict or None, optional See `fit` for a description of required keywords for alternative covariance estimators return_raw : bool,optional Whether or not to return only the raw Hamilton filter output or a full results object. Default is to return a full results object. results_class : type, optional A results class to instantiate rather than `MarkovSwitchingResults`. Usually only used internally by subclasses. results_wrapper_class : type, optional A results wrapper class to instantiate rather than `MarkovSwitchingResults`. Usually only used internally by subclasses. Returns ------- MarkovSwitchingResults rr rVrUrWr]r^r_r`rerfrq) r%r&transform_params param_namesdataHamiltonFilterResultsr dictzipr _wrap_results) r|r transformedcov_typecov_kwdsr results_classresults_wrapper_classnamesresults r4rzMarkovSwitching.filtersD&*** 3**622F!% 0 555' %AA$s5$,,v*>*>??@@AACC!!&&*h"*M"799 9r6cR|||}t|||Sr)rrl)r|rrerfrVs r4_smoothzMarkovSwitching._smoothKs>  $ $ = =f E E ##4#D#CEE Er6c"dttfiS)Nfit)MarkovSwitchingResultsMarkovSwitchingResultsWrapperrs r4 _res_classeszMarkovSwitching._res_classesVs.0MNOOr6c|sPi}|||d<|||d<||jdd}||jdd}||||fi|} || }|S)Nrrrrr)r) r|rrrrrr wrapper_class result_kwargsress r4rzMarkovSwitching._wrap_resultsZs (M#,4 j)#,4 j)$ $ 1% 8 ; $ $ 1% 8 ; -ffFF FFC"]3''F r6c tj|d}|s||}|j|j_gd}t dit t|||} | || j | j } | d| d<| d| d<t|| } | || |||||S)a Apply the Kim smoother and Hamilton filter Parameters ---------- params : array_like Array of parameters at which to perform filtering. transformed : bool, optional Whether or not `params` is already transformed. Default is True. cov_type : str, optional See `fit` for a description of covariance matrix types for results object. cov_kwds : dict or None, optional See `fit` for a description of required keywords for alternative covariance estimators return_raw : bool,optional Whether or not to return only the raw Hamilton filter output or a full results object. Default is to return a full results object. results_class : type, optional A results class to instantiate rather than `MarkovSwitchingResults`. Usually only used internally by subclasses. results_wrapper_class : type, optional A results wrapper class to instantiate rather than `MarkovSwitchingResults`. Usually only used internally by subclasses. Returns ------- MarkovSwitchingResults rrrrrjrkrq)r%r&rrrr rrrrrerfKimSmootherResultsr) r|rrrrrrrrrouts r4rzMarkovSwitching.smoothmsD&*** 3**622F!% 0  555@@c%f)=)=>>??@@ll66#K!BDD14Q-.47F01#D&11!!&&*h"*M"799 9r6ctj|d}|s||}||}|dS)aL Loglikelihood evaluation for each period Parameters ---------- params : array_like Array of parameters at which to evaluate the loglikelihood function. transformed : bool, optional Whether or not `params` is already transformed. Default is True. rr)r%r&rr)r|rrrs r4 loglikeobszMarkovSwitching.loglikeobssN&*** 3**622F,,v&&qzr6cRtj|||S)a< Loglikelihood evaluation Parameters ---------- params : array_like Array of parameters at which to evaluate the loglikelihood function. transformed : bool, optional Whether or not `params` is already transformed. Default is True. )r%rTrr|rrs r4loglikezMarkovSwitching.loglikes"vdoofk::;;;r6c^tj|d}t||j|fS)aA Compute the score function at params. Parameters ---------- params : array_like Array of parameters at which to evaluate the score function. transformed : bool, optional Whether or not `params` is already transformed. Default is True. rrargs)r%r&r r r s r4scorezMarkovSwitching.scores1&*** K>JJJJr6c^tj|d}t||j|fS)aR Compute the score per observation, evaluated at params Parameters ---------- params : array_like Array of parameters at which to evaluate the score function. transformed : bool, optional Whether or not `params` is already transformed. Default is True. rrr )r%r&r rr s r4 score_obszMarkovSwitching.score_obss1&***{nMMMMr6cXtj|d}t||jS)ar Hessian matrix of the likelihood function, evaluated at the given parameters Parameters ---------- params : array_like Array of parameters at which to evaluate the Hessian function. transformed : bool, optional Whether or not `params` is already transformed. Default is True. rr)r%r&r r r s r4hessianzMarkovSwitching.hessians*&***fdl333r6approxbfgsdrr?c | |j}d}ntj|d}| dkr|| ||| |}d}| r#|js|||| dd}d}|r||}d}tj|f|||||| dd |}| r| |j }n=| |j d || }||_ |j |_ |j|_|S) a Fits the model by maximum likelihood via Hamilton filter. Parameters ---------- start_params : array_like, optional Initial guess of the solution for the loglikelihood maximization. If None, the default is given by Model.start_params. transformed : bool, optional Whether or not `start_params` is already transformed. Default is True. cov_type : str, optional The type of covariance matrix estimator to use. Can be one of 'approx', 'opg', 'robust', or 'none'. Default is 'approx'. cov_kwds : dict or None, optional Keywords for alternative covariance estimators method : str, optional The `method` determines which solver from `scipy.optimize` is used, and it can be chosen from among the following strings: - 'newton' for Newton-Raphson, 'nm' for Nelder-Mead - 'bfgs' for Broyden-Fletcher-Goldfarb-Shanno (BFGS) - 'lbfgs' for limited-memory BFGS with optional box constraints - 'powell' for modified Powell's method - 'cg' for conjugate gradient - 'ncg' for Newton-conjugate gradient - 'basinhopping' for global basin-hopping solver The explicit arguments in `fit` are passed to the solver, with the exception of the basin-hopping solver. Each solver has several optional arguments that are not the same across solvers. See the notes section below (or scipy.optimize) for the available arguments and for the list of explicit arguments that the basin-hopping solver supports. maxiter : int, optional The maximum number of iterations to perform. full_output : bool, optional Set to True to have all available output in the Results object's mle_retvals attribute. The output is dependent on the solver. See LikelihoodModelResults notes section for more information. disp : bool, optional Set to True to print convergence messages. callback : callable callback(xk), optional Called after each iteration, as callback(xk), where xk is the current parameter vector. return_params : bool, optional Whether or not to return only the array of maximizing parameters. Default is False. em_iter : int, optional Number of initial EM iteration steps used to improve starting parameters. search_reps : int, optional Number of randomly drawn search parameters that are drawn around `start_params` to try and improve starting parameters. Default is 0. search_iter : int, optional Number of initial EM iteration steps used to improve each of the search parameter repetitions. search_scale : float or array, optional. Scale of variates for random start parameter search. **kwargs Additional keyword arguments to pass to the optimizer. Returns ------- MarkovSwitchingResults NTrrr) start_paramsrem_iterscale)rmaxiter tolerance return_params)F)methodfargsr full_outputdispcallback skip_hessianFrrr)rr%r&_start_params_searchr_fit_emuntransform_paramsrrrrrmlefit mle_retvals mle_settings)r|rrrrrrr!r"r#rr search_reps search_iter search_scalekwargsr r)rrs r4rzMarkovSwitching.fitsuP  ,LKK8L:::L ??44,'"5$$LK  49 << +0716:(<c  | |j}d}ntj|d}|s||}g} |g} d} d} | |kr| dks| |kr|| d}| |dj| |d| dkr8d| d| dz ztj| d| dzz } | dz } | |kr | dk| |k|r | d}nw|| dd|| }|rDtd itj| tj| | d }td i||d }nd}d}||_ ||_ |S) a Fits the model using the Expectation-Maximization (EM) algorithm Parameters ---------- start_params : array_like, optional Initial guess of the solution for the loglikelihood maximization. If None, the default is given by `start_params`. transformed : bool, optional Whether or not `start_params` is already transformed. Default is True. cov_type : str, optional The type of covariance matrix estimator to use. Can be one of 'approx', 'opg', 'robust', or 'none'. Default is 'none'. cov_kwds : dict or None, optional Keywords for alternative covariance estimators maxiter : int, optional The maximum number of iterations to perform. tolerance : float, optional The iteration stops when the difference between subsequent loglikelihood values is less than this tolerance. full_output : bool, optional Set to True to have all available output in the Results object's mle_retvals attribute. This includes all intermediate values for parameters and loglikelihood values return_params : bool, optional Whether or not to return only the array of maximizing parameters. Default is False. **kwargs Additional keyword arguments to pass to the optimizer. Notes ----- This is a private method for finding good starting parameters for MLE by scoring. It has not been tested for a thoroughly correct EM implementation in all cases. It does not support TVTP transition probabilities. Returns ------- MarkovSwitchingResults NTrrrr r"rNr%)rllfiter)rrrq) rr%r&r _em_iterationrr3rrr r*r+)r|rrrrrrr!rr/r3rrFdeltarr em_retvals em_settingss r4r'zMarkovSwitching._fit_emzs\  ,LKK8L:::L ?00>>L 'kkq1uu):):$$VBZ00C JJs1vz " " " MM#a& ! ! !1uuSWs2w./"&R3r79J2K2KK FA 'kkq1uu):):  .BZFF[[*2X!GGF #"220@0@-/Xc]]./&1&122 $<** * *%a(+>(>?@@wr6cj|j}t|jdz D]}tj|d}|}t |jd}tj|j|f}t|jD]}t|jdz D]C}tj|||ftj|j |z |||f<Dtj||dz }|dkr0tj dt||xxd|zdzzcc<|S)z= EM step for regime transition probabilities r!rN)rrVrrz|Invalid regime transition probabilities estimated in EM iteration; probabilities have been re-scaled to continue estimation.r1) rjrBr'r%rTr)rr-rYrkwarningswarnr ) r|rr2rFrjrrVrAr6s r4r:z%MarkovSwitching._em_regime_transitions\ 1sx!|$$ " "A&b//CC'*$4?+ABCC Hdnl%CDDt~&& 9 9A4>A-.. G GF71=>>F6A!DEEF"!Q$''F,Q/0014Eqyy H0111"!$$$E D(88$$$  r6c| |j}d}ntj|d}|r||}tj|d}|jdkrtj|j|z}|j|jkstdtj||jf}t|jD]4}||tj dd|z|dd|f<5| |d }|} t|D]}tj5tjd  ||||zd |d } | | } | |kr| }|| } n#t"$rYnwxYwdddn #1swxYwY|| S) a Search for starting parameters as random permutations of a vector Parameters ---------- reps : int Number of random permutations to try. start_params : ndarray, optional Starting parameter vector. If not given, class-level start parameters are used. transformed : bool, optional If `start_params` was provided, whether or not those parameters are already transformed. Default is True. em_iter : int, optional Number of EM iterations to apply to each random permutation. scale : array or float, optional Scale of variates for random start parameter search. Can be given as an array of length equal to the number of parameters or as a single scalar. Notes ----- This is a private method for finding good starting parameters for MLE by scoring, where the defaults have been set heuristically. NTrrzkScale of variates for random start parameter search must be given for each parameter or as a single scalar.gg?)rFrignore)rrr)rr%r&r(rrrurOr-rBrandomuniformr r>catch_warnings simplefilterr' Exceptionr) r|repsrrrrvariatesrFr3rproposed_params proposed_llfs r4r&z$MarkovSwitching._start_params_searchsT6  ,LKK8L:::L  A22<@@La((( :??GDM**U2EzT]**ABB B 8T4=122t}%% P PA"1X (9(9$$(9(O(OOHQQQTNNll AF"!F>" F/,F>.F//F>>G G ctj|jtj}|jrd||jd<nd|jz ||jd<|S)zP (array) Starting parameters for maximum likelihood estimation. rJgrVr)r%r-rurrrrYrs r4rzMarkovSwitching.start_params\s\ $-rz::: 9 O;=F4?#67 8 8;=;NF4?#67 8 r6c@tjjt}jr2fdt jdz D|jd<n1fdt jdz D|jd<|S)z (list of str) List of human readable parameter names (for parameters actually included in the model). rJc|g|]8}tjD]!}tjD] }d|||fz "9S)zp[%d->%d].tvtp%d)rBrrY)r<rFrrAr|s r4r>z/MarkovSwitching.param_names..vsAAAt{++AAt~.. AA#aAY.AAAAAr6rrVcLg|] }tjD] }d||fz !S)z p[%d->%d])rBrY)r<rFrAr|s r4r>z/MarkovSwitching.param_names..}s\A0A0A0t~..A0A0q!f$A0A0A0A0r6) r%r-ruobjectrrBrYrr)r|rs` r4rzMarkovSwitching.param_namesks ht}F;;;  9 0AAAAt~a/00AAAK(;< = =A0A0A0A0t~a/00A0A0A0K(;< = !!###r6ctj|d}|tjtj|j}|jr"||jd||jd<ntt|j D]_}||j|df}tj d|f}tj |t|z ||j|df<`|S)ac Transform unconstrained parameters used by the optimizer to constrained parameters used in likelihood evaluation Parameters ---------- unconstrained : array_like Array of unconstrained parameters used by the optimizer, to be transformed. Returns ------- constrained : array_like Array of constrained parameters which may be used in likelihood evaluation. Notes ----- In the base class, this only transforms the transition-probability- related parameters. TrRrVr) r%r&rrrrKrrrBrYrr/r)r| unconstrained constrainedrFtmp1tmp2s r4rz MarkovSwitching.transform_paramss,h}4888 !((  RZ): ; ;==  9 ,do.ABC (;< = = 4>** , ,$T_Q8K5K%LMuQW~GIv9T??*H,H, DOA/B,BCDD r6cntj|j|j}tj|}tj|}t t|D]O}||tjd|z||z z tjd||z dz z||<P|S)zl Function to allow using a numerical root-finder to reverse the logistic transform. rJr) r%r-r*rKr/rTrBr)rP)r|rSrTresidr/sum_exprFs r4_untransform_logisticz%MarkovSwitching._untransform_logistics ,M4GHHHf]##&++s=))** 8 8A%a(q7{SV3445q;q>1A5667E!HH r6chtj|d}|tjtj|j}|jr"||jd||jd<nt|j D]}|j|df}|j dkr%tj d||z dz  ||<Addl m }||j tj||j|j||f }|d st!d |d ||<|S) aL Transform constrained parameters used in likelihood evaluation to unconstrained parameters used by the optimizer Parameters ---------- constrained : array_like Array of constrained parameters used in likelihood evaluation, to be transformed. Returns ------- unconstrained : array_like Array of unconstrained parameters used by the optimizer. Notes ----- In the base class, this only untransforms the transition-probability- related parameters. TrRrVr rrr)rootr successz!Could not untransform parameters.r0)r%r&rrrrKrrrBrYrPscipy.optimizer\rZr-r*rO)r|rTrSrFrr\rs r4r(z"MarkovSwitching.untransform_paramssb*4888 %,,  RZ)< = =??  9 0DO,?@A $/*=> ? ?4>** 0 0OA':$:;>Q&&(*rKN/BQ/F(G(G'GM!$$333333$t9!x a(8(>(5(; = =%0^$5777Cy>N()LMMM'*3xM!$$r6)rNNNNr)rrNNNF)TNNFNN)NNNNT)NTrNrrrrNFrrrr)NTrNr0r1TF)NTrr)%rrrrr}propertyrurrrUrrrrrrrrrrrrr rrrrr'r5r:r&rrrrZr( __classcell__rs@r4rrs3@HL06*+*+*+*+*+*+X((X( + + + 4 4 4 48((((8 ( ( ( (DCG!H6H6H6H6T"""$"""====$HL/3%)89898989vEI E E E EPPXPBFGK&HL/3%)@9@9@9@9D* < < < <KKKK NNNN 4444"AIKLGH(*uuuuuunEKGK#ZZZZx8 ! ! !DIM.0F-F-F-F-P  X $$X$0)))V   ///////r6rcDeZdZdZdZedZedZdS)ra Results from applying the Hamilton filter to a state space model. Parameters ---------- model : Representation A Statespace representation Attributes ---------- nobs : int Number of observations. k_endog : int The dimension of the observation series. k_regimes : int The number of unobserved regimes. regime_transition : ndarray The regime transition matrix. initialization : str Initialization method for regime probabilities. initial_probabilities : ndarray Initial regime probabilities conditional_loglikelihoods : ndarray The loglikelihood values at each time period, conditional on regime. predicted_joint_probabilities : ndarray Predicted joint probabilities at each time period. filtered_marginal_probabilities : ndarray Filtered marginal probabilities at each time period. filtered_joint_probabilities : ndarray Filtered joint probabilities at each time period. joint_loglikelihoods : ndarray The likelihood values at each time period. llf_obs : ndarray The loglikelihood values at each time period. c ||_|j|_|j|_|j|_gd}|D]!}t ||t ||"|j|_|j|_ tj |j |_ |j jddkr&|jdkr|j d|jdf|_ d|_dS)N)rVrUrWr^r]r`r_r"rr.)modelrZr[rYsetattrgetattrrinitializationr_llf_obsr%rTr3rVr*!_predicted_marginal_probabilities)r|rfr attributesnames r4r}zHamiltonFilterResults.__init__s J [ ...  7 7D D$ 5 5 6 6 6 6#30 6$,''  ! ' +a / /DJNN%)%;CCL ' ' ]h & &%(%:C "*6x*@*G*G>>r6c@|j|jSzY (float) The value of the log-likelihood function evaluated at `params`. )rfrrrs r4rjzMarkovSwitchingResults.llf_obsgs z$$T[111r6c@|j|jSr)rfr rrs r4r3zMarkovSwitchingResults.llfnrr6c*|jj|jz S)zI (array) The model residuals. An (nobs x k_endog) array. )rfrrrs r4rXzMarkovSwitchingResults.residus z$"333r6c4tj|jSr)r%r/r_rs r4joint_likelihoodsz(MarkovSwitchingResults.joint_likelihoods|svd/000r6cJ|j|j||||S)a In-sample prediction and out-of-sample forecasting Parameters ---------- start : int, str, or datetime, optional Zero-indexed observation number at which to start forecasting, i.e., the first forecast is start. Can also be a date string to parse or a datetime type. Default is the the zeroth observation. end : int, str, or datetime, optional Zero-indexed observation number at which to end forecasting, i.e., the last forecast is end. Can also be a date string to parse or a datetime type. However, if the dates index does not have a fixed frequency, end must be an integer index if you want out of sample prediction. Default is the last observation in the sample. probabilities : str or array_like, optional Specifies the weighting probabilities used in constructing the prediction as a weighted average. If a string, can be 'predicted', 'filtered', or 'smoothed'. Otherwise can be an array of probabilities to use. Default is smoothed. conditional : bool or int, optional Whether or not to return predictions conditional on current or past regimes. If False, returns a single vector of weighted predictions. If True or 1, returns predictions conditional on the current regime. For larger integers, returns predictions conditional on the current regime and some number of past regimes. Returns ------- predict : ndarray Array of out of in-sample predictions and / or out-of-sample forecasts. An (npredict x k_endog) array. )rrrrr)r|rrrrs r4rzMarkovSwitchingResults.predicts2Hz!!$+U0=.9";; ;r6rc t)a Out-of-sample forecasts Parameters ---------- steps : int, str, or datetime, optional If an integer, the number of steps to forecast from the end of the sample. Can also be a date string to parse or a datetime type. However, if the dates index does not have a fixed frequency, steps must be an integer. Default **kwargs Additional arguments may required for forecasting beyond the end of the sample. See `FilterResults.predict` for more details. Returns ------- forecast : ndarray Array of out of sample forecasts. A (steps x k_endog) array. r)r|stepsr/s r4forecastzMarkovSwitchingResults.forecasts ("!r6皙?Tc4 ddlm}|j}|d}|d}|jjS|jj}||} d| j| j| jfzg} |d} | dd| j| j| jfzzgz } n+t|dt|jj zg} | |j j }t|ts|g}d g} | d |dgftd t!|D]#} | d d || zgf$| ddd| dgfd | d gfgz } d|jj gfdd|jzgfdd|jzgfdd|jzgfdd|jzgfg} t+|dr| d|jgf|}||| | |ddlddlmd&fd }|j}dt|jD}i}|jD]\}}t!|}|dkrg||<t|D]t} || rAt|jD]+}|||||f| ,K|||d|f| ut|jD]G} || }t!|dkr*|||d| z}|j|Hg}|D]\}}||t!|dkr'|||d }|j||d}|||d!}|j|g}t+|dr)d"|j vr ||j d"|j!t!|j"kr7|d#tG|$z|rDd$tK|D}|&dd%|'||S)'ay Summarize the Model Parameters ---------- alpha : float, optional Significance level for the confidence intervals. Default is 0.05. start : int, optional Integer of the start observation. Default is 0. title : str, optional The title of the summary table. model_name : str The name of the model used. Default is to use model class name. display_params : bool, optional Whether or not to display tables of estimated parameters. Default is True. Usually only used internally. Returns ------- summary : Summary instance This holds the summary table and text, which can be printed or converted to various output formats. See Also -------- statsmodels.iolib.summary.Summary r)SummaryNzMarkov Switching Model Resultsz%02d-%02d-%02dr"z- z - )zDep. Variable:NzModel:rz+ )zDate:N)zTime:NzSample:zNo. Observations:zLog Likelihoodz%#5.3fAICBICHQICrzCovariance Type:)gleftgrighttitle)summary_paramsTcD||j||j||j||j|||f}fdt j|jj| D}|d|d|S)Nc>g|]}dd|S)z\[\d+\]$r)sub)r<rmres r4r>zFMarkovSwitchingResults.summary..make_table..s626{B--r6F)ynamexnamealphause_tr) rbsetvaluespvaluesconf_intr%r&rrr) r|maskr strip_endrrrrrs r4 make_tablez2MarkovSwitchingResults.summary..make_tablesT*DHTN<%t|D'9==''-/C.//5<<>>K ">#T(-U%III Ir6cg|]}gSrqrqrrs r4r>z2MarkovSwitchingResults.summary.."s;;;q;;;r6rVzRegime %d parameterszNon-switching parameterszRegime transition parametersrzmCovariance matrix is singular or near-singular, with condition number %6.3g. Standard errors may be unstable.c*g|]\}}d|dzd|S)[rz] rq)r<rFtexts r4r>z2MarkovSwitchingResults.summary..Ns@666 D)Q(($((666r6z Warnings:r`)(statsmodels.iolib.summaryrrfrrmonthdayyearrrZrrrlistrrBr)r3rrrrradd_table_2colsrrrrYrwrtablesextendrrrrr enumerateinsert add_extra_txt)r|rrr model_namedisplay_paramsrrfrrsampletop_leftrF top_rightsummaryrr regime_masks other_masksrrwrurArtable_masketextrrs ` @@r4rzMarkovSwitchingResults.summarys: 655555  =4E =E 9? &IOEe A&!'15!&)AABFb A t.!'15!&1IIIJ JFF%jj%#djo*>*>">?F  1J*d++ &$J,-JqM?3444q#j//** : :A OOR$A"6!78 9 9 9 9    $ &)    !4:?"3 4 48 34 5 X() * X() * h*+ ,   4 $ $ C OO/$-A B B B'))HY&+  - - -  <<<<<< I I I I I I I I!;;E%/$:$:;;;  $.4466 ? ?NC9~~H)))!K 8__ ? ?Q<?"4>22BB$Q..vaf~a/@AAAAB $++F1c6N1,=>>>>  ?t~&& - -A?D4yy1}}" 4/E/IJJ%%e,,,%++--  JC KK     t99q==JtT+EFFE N ! !% ( ( ()* 4'EFFe$$$ 4 $ $ 7$-)G)G LL}5 6 6 6 :DK(( ( ( LL&(24??3D3D(E(EF G G G  )66$-e$4$4666E LLK ( ( (  ! !% ( ( (r6)r{N)r{r_rL)rNNNT)rrrrrr}rrrrrrrrrrjr3rXrarrrrrqr6r4rrps8 EVGVGVGVGp1111f>>^>>>^>  ^   ^ ^ //^/ ??^?22^2 //^/ 44^4 11X1;?!&;&;&;&;P"""",EI#UUUUUUr6rceZdZdddddZejejjeZddiZ ejejj e Z dS)rcov)rrrrrrN) rrr_attrswrap union_dictsrTimeSeriesResultsWrapper _wrap_attrs_methods _wrap_methodsrqr6r4rrVsx"#" F #$"6#B#N#)++K GH%D$'5xAAMMMr6r)>rr>numpyr%pandasr scipy.specialrstatsmodels.base.datarstatsmodels.base.wrapperbasewrapperrstatsmodels.tools.decoratorsrstatsmodels.tools.eval_measuresrrrstatsmodels.tools.numdiffr r statsmodels.tools.sm_exceptionsr statsmodels.tools.toolsr r statsmodels.tsa.base.tsa_modeltsa tsa_modelr1statsmodels.tsa.regime_switching._hamilton_filterrrrr.statsmodels.tsa.regime_switching._kim_smootherrrrr statsmodels.tsa.statespace.toolsrrrrSrir5rGrgrlrnTimeSeriesModelrrrrrResultsWrapperrpopulate_wrapperrqr6r4rs ######,,,,,,'''''''''777777::::::::::FFFFFFFF======88888888////////////   $8 $8"" !2 !2 4Dy7y7y7xGIGIGITY.Y.Y.Y.Y.Y.Y.Y.x\\\\\f,\\\~ _,_,_,_,_,_,_,_,D77777.777>cccccV:cccL A A A A AD$7 A A A3,.....r6