M/Ph{EdZddlZddlmZddlmZddlmZm Z ddl m Z m Z m Z ddlmZmZdZd Zd Zd Zd ZeezezezezZdZd Zd Zd ZGd deZGdde ZdS)ze State Space Representation and Kalman Filter, Smoother Author: Chad Fulton License: Simplified-BSD N)SimpleNamespace) OptionWrapper) KalmanFilter FilterResults)reorder_missing_matrixreorder_missing_vectorcopy_index_matrix)toolsinitializationceZdZdZgdZedeZedeZ ede Z ede Z edeZedeZgdZedeZ edeZ edeZ edeZ eZdZ dfd Zfd Zed Z dd Z!dd Z"dd Z# ddZ$ ddZ%xZ&S)KalmanSmoothera State space representation of a time series process, with Kalman filter and smoother. Parameters ---------- k_endog : {array_like, int} The observed time-series process :math:`y` if array like or the number of variables in the process if an integer. k_states : int The dimension of the unobserved state process. k_posdef : int, optional The dimension of a guaranteed positive definite covariance matrix describing the shocks in the measurement equation. Must be less than or equal to `k_states`. Default is `k_states`. results_class : class, optional Default results class to use to save filtering output. Default is `SmootherResults`. If specified, class must extend from `SmootherResults`. **kwargs Keyword arguments may be used to provide default values for state space matrices, for Kalman filtering options, or for Kalman smoothing options. See `Representation` for more details. )smoother_statesmoother_state_covsmoother_state_autocovsmoother_disturbancesmoother_disturbance_cov smoother_allsmoother_output)smooth_conventionalsmooth_alternativesmooth_classical smooth_methodrNc `|t}dgtjz}fd|D}dgtjz}fd|D} t j|||fd|i||nt j|_i|_ |j di||j di| dS)NrcDi|]}|v||Spop.0keykwargss j/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/tsa/statespace/kalman_smoother.py z+KalmanSmoother.__init__..os5"4"4"43%(F]]#&vzz#%2]]rcDi|]}|v||Sr r!r#s r'r(z+KalmanSmoother.__init__..rs5 2 2 2#&&==!$VZZ__#0==r) results_classr ) SmootherResultsrsmoother_outputssmooth_methodssuper__init__r prefix_kalman_smoother_mapcopy_kalman_smoothersset_smoother_outputset_smooth_method) selfk_endogk_statesk_posdefr+kalman_smoother_classesr&keyssmoother_output_kwargssmooth_method_kwargs __class__s ` r'r0zKalmanSmoother.__init__gs,  +M""^%DD"4"4"4"4$"4"4"4 >#@@ 2 2 2 2 2 2 2  Xx  7D HN   '2 $ #16688 ' "$ ! ::#9:::66!566666r)c tj|fi|}|d|j|d|j|S)Nrr)r/ _clone_kwargs setdefaultrr)r6endogr&r>s r'r@zKalmanSmoother._clone_kwargss[&&u7777 +T-ABBB/4+=>>> r)c@|j}||jvr |j|SdSN)prefixr3)r6rEs r'_kalman_smootherzKalmanSmoother._kalman_smoothers* T+ + +)&1 1tr)c ||j}||j}|j|fi|\}}}}|p||jv}|s!|j|} | j|j|u}|r:|j|} | |j||j||||j|<nA|j||d|j| ||||||fS)NF) rr_initialize_filterr3kfilter_kalman_filtersr1 _statespacesr4r5) r6rrrEr&dtype create_filtercreate_statespacecreate_smootherkalman_smootherclss r'_initialize_smootherz#KalmanSmoother._initialize_smoothers1  ""2O   .M $D #F 5 5f 5 5 8}&7 )@!)??  ="4V > ( ( (  "6 * < <] K K Kuo}>OOOr)c l|||_tjD]}||vrt||||dS)aV Set the smoother output The smoother can produce several types of results. The smoother output variable controls which are calculated and returned. Parameters ---------- smoother_output : int, optional Bitmask value to set the smoother output to. See notes for details. **kwargs Keyword arguments may be used to influence the smoother output by setting individual boolean flags. See notes for details. Notes ----- The smoother output is defined by a collection of boolean flags, and is internally stored as a bitmask. The methods available are: SMOOTHER_STATE = 0x01 Calculate and return the smoothed states. SMOOTHER_STATE_COV = 0x02 Calculate and return the smoothed state covariance matrices. SMOOTHER_STATE_AUTOCOV = 0x10 Calculate and return the smoothed state lag-one autocovariance matrices. SMOOTHER_DISTURBANCE = 0x04 Calculate and return the smoothed state and observation disturbances. SMOOTHER_DISTURBANCE_COV = 0x08 Calculate and return the covariance matrices for the smoothed state and observation disturbances. SMOOTHER_ALL Calculate and return all results. If the bitmask is set directly via the `smoother_output` argument, then the full method must be provided. If keyword arguments are used to set individual boolean flags, then the lowercase of the method must be used as an argument name, and the value is the desired value of the boolean flag (True or False). Note that the smoother output may also be specified by directly modifying the class attributes which are defined similarly to the keyword arguments. The default smoother output is SMOOTHER_ALL. If performance is a concern, only those results which are needed should be specified as any results that are not specified will not be calculated. For example, if the smoother output is set to only include SMOOTHER_STATE, the smoother operates much more quickly than if all output is required. Examples -------- >>> import statsmodels.tsa.statespace.kalman_smoother as ks >>> mod = ks.KalmanSmoother(1,1) >>> mod.smoother_output 15 >>> mod.set_smoother_output(smoother_output=0) >>> mod.smoother_state = True >>> mod.smoother_output 1 >>> mod.smoother_state True N)rrr-setattr)r6rr&names r'r4z"KalmanSmoother.set_smoother_outputsOH  D "3 2 2Dv~~dF4L111 2 2r)c l|||_tjD]}||vrt||||dS)a Set the smoothing method The smoothing method can be used to override the Kalman smoother approach used. By default, the Kalman smoother used depends on the Kalman filter method. Parameters ---------- smooth_method : int, optional Bitmask value to set the filter method to. See notes for details. **kwargs Keyword arguments may be used to influence the filter method by setting individual boolean flags. See notes for details. Notes ----- The smoothing method is defined by a collection of boolean flags, and is internally stored as a bitmask. The methods available are: SMOOTH_CONVENTIONAL = 0x01 Default Kalman smoother, as presented in Durbin and Koopman, 2012 chapter 4. SMOOTH_CLASSICAL = 0x02 Classical Kalman smoother, as presented in Anderson and Moore, 1979 or Durbin and Koopman, 2012 chapter 4.6.1. SMOOTH_ALTERNATIVE = 0x04 Modified Bryson-Frazier Kalman smoother method; this is identical to the conventional method of Durbin and Koopman, 2012, except that an additional intermediate step is included. SMOOTH_UNIVARIATE = 0x08 Univariate Kalman smoother, as presented in Durbin and Koopman, 2012 chapter 6, except with modified Bryson-Frazier timing. Practically speaking, these methods should all produce the same output but different computational implications, numerical stability implications, or internal timing assumptions. Note that only the first method is available if using a Scipy version older than 0.16. If the bitmask is set directly via the `smooth_method` argument, then the full method must be provided. If keyword arguments are used to set individual boolean flags, then the lowercase of the method must be used as an argument name, and the value is the desired value of the boolean flag (True or False). Note that the filter method may also be specified by directly modifying the class attributes which are defined similarly to the keyword arguments. The default filtering method is SMOOTH_CONVENTIONAL. Examples -------- >>> mod = sm.tsa.statespace.SARIMAX(range(10)) >>> mod.smooth_method 1 >>> mod.filter_conventional True >>> mod.filter_univariate = True >>> mod.smooth_method 17 >>> mod.set_smooth_method(filter_univariate=False, filter_collapsed=True) >>> mod.smooth_method 33 >>> mod.set_smooth_method(smooth_method=1) >>> mod.filter_conventional True >>> mod.filter_univariate False >>> mod.filter_collapsed False >>> mod.filter_univariate = True >>> mod.smooth_method 17 N)rrr.rT)r6rr&rUs r'r5z KalmanSmoother.set_smooth_method sO`  $!.D "1 2 2Dv~~dF4L111 2 2r)Fc |j||fd|i|\}}}} } | s| rtd|j|} | | S)NrEzgPassed settings forced re-creation of the Kalman filter. Please run `_filter` before running `_smooth`.)rR ValueErrorr3) r6rrrE complex_stepresultsr&rLrOrMrNsmoothers r'_smoothzKalmanSmoother._smooth_s &D %  7= AG   I 7H  4- 4344 4 )&1  r)Tc  |jdi| } ||}|r|||r|| n | j|_||j}|j|fd|i| } | r|| |S)a Apply the Kalman smoother to the statespace model. Parameters ---------- smoother_output : int, optional Determines which Kalman smoother output calculate. Default is all (including state, disturbances, and all covariances). results : class or object, optional If a class, then that class is instantiated and returned with the result of both filtering and smoothing. If an object, then that object is updated with the smoothing data. If None, then a SmootherResults object is returned with both filtering and smoothing results. run_filter : bool, optional Whether or not to run the Kalman filter prior to smoothing. Default is True. prefix : str The prefix of the datatype. Usually only used internally. Returns ------- SmootherResults object NrZr )_filterr+update_representation update_filter nobs_diffuserr\update_smoother) r6rrrZ run_filterrErYr_r`rbr&rIr[s r'smoothzKalmanSmoother.smoothus<$,(((($$T**  0  ) )$ / / /  8  ! !' * * * *$+#7G   ""2O4<KKKFKK  .  # #H - - -r))NNNrD)NNNFN) NNNTNFTTT)'__name__ __module__ __qualname____doc__r-rSMOOTHER_STATErSMOOTHER_STATE_COVrSMOOTHER_DISTURBANCErSMOOTHER_DISTURBANCE_COVrSMOOTHER_STATE_AUTOCOVr SMOOTHER_ALLrr.SMOOTH_CONVENTIONALrSMOOTH_ALTERNATIVErSMOOTH_CLASSICALrSMOOTH_UNIVARIATEsmooth_univariaterrr0r@propertyrFrRr4r5r\rd __classcell__r>s@r'rr"s22 #]#4nEEN&'8:LMM ')=>>  ')ABB  ')?@@!=!2LAALN(-9LMM'8JKK%}_6FGG& o7HII #OMGK)-777777@X HL$(%P%P%P%PNH2H2H2H2TT2T2T2T2lHL,0,HL:?9=#44444444r)rceZdZdZgdZejZej ezZ dfd Z dZ ddZ dd Z dd Z dd Zd ZedZedZedZ ddZxZS)r,aG Results from applying the Kalman smoother and/or filter to a state space model. Parameters ---------- model : Representation A Statespace representation Attributes ---------- nobs : int Number of observations. k_endog : int The dimension of the observation series. k_states : int The dimension of the unobserved state process. k_posdef : int The dimension of a guaranteed positive definite covariance matrix describing the shocks in the measurement equation. dtype : dtype Datatype of representation matrices prefix : str BLAS prefix of representation matrices shapes : dictionary of name:tuple A dictionary recording the shapes of each of the representation matrices as tuples. endog : ndarray The observation vector. design : ndarray The design matrix, :math:`Z`. obs_intercept : ndarray The intercept for the observation equation, :math:`d`. obs_cov : ndarray The covariance matrix for the observation equation :math:`H`. transition : ndarray The transition matrix, :math:`T`. state_intercept : ndarray The intercept for the transition equation, :math:`c`. selection : ndarray The selection matrix, :math:`R`. state_cov : ndarray The covariance matrix for the state equation :math:`Q`. missing : array of bool An array of the same size as `endog`, filled with boolean values that are True if the corresponding entry in `endog` is NaN and False otherwise. nmissing : array of int An array of size `nobs`, where the ith entry is the number (between 0 and k_endog) of NaNs in the ith row of the `endog` array. time_invariant : bool Whether or not the representation matrices are time-invariant initialization : str Kalman filter initialization method. initial_state : array_like The state vector used to initialize the Kalamn filter. initial_state_cov : array_like The state covariance matrix used to initialize the Kalamn filter. filter_method : int Bitmask representing the Kalman filtering method inversion_method : int Bitmask representing the method used to invert the forecast error covariance matrix. stability_method : int Bitmask representing the methods used to promote numerical stability in the Kalman filter recursions. conserve_memory : int Bitmask representing the selected memory conservation method. tolerance : float The tolerance at which the Kalman filter determines convergence to steady-state. loglikelihood_burn : int The number of initial periods during which the loglikelihood is not recorded. converged : bool Whether or not the Kalman filter converged. period_converged : int The time period in which the Kalman filter converged. filtered_state : ndarray The filtered state vector at each time period. filtered_state_cov : ndarray The filtered state covariance matrix at each time period. predicted_state : ndarray The predicted state vector at each time period. predicted_state_cov : ndarray The predicted state covariance matrix at each time period. kalman_gain : ndarray The Kalman gain at each time period. forecasts : ndarray The one-step-ahead forecasts of observations at each time period. forecasts_error : ndarray The forecast errors at each time period. forecasts_error_cov : ndarray The forecast error covariance matrices at each time period. loglikelihood : ndarray The loglikelihood values at each time period. collapsed_forecasts : ndarray If filtering using collapsed observations, stores the one-step-ahead forecasts of collapsed observations at each time period. collapsed_forecasts_error : ndarray If filtering using collapsed observations, stores the one-step-ahead forecast errors of collapsed observations at each time period. collapsed_forecasts_error_cov : ndarray If filtering using collapsed observations, stores the one-step-ahead forecast error covariance matrices of collapsed observations at each time period. standardized_forecast_error : ndarray The standardized forecast errors smoother_output : int Bitmask representing the generated Kalman smoothing output scaled_smoothed_estimator : ndarray The scaled smoothed estimator at each time period. scaled_smoothed_estimator_cov : ndarray The scaled smoothed estimator covariance matrices at each time period. smoothing_error : ndarray The smoothing error covariance matrices at each time period. smoothed_state : ndarray The smoothed state at each time period. smoothed_state_cov : ndarray The smoothed state covariance matrices at each time period. smoothed_state_autocov : ndarray The smoothed state lago-one autocovariance matrices at each time period: :math:`Cov(\alpha_{t+1}, \alpha_t)`. smoothed_measurement_disturbance : ndarray The smoothed measurement at each time period. smoothed_state_disturbance : ndarray The smoothed state at each time period. smoothed_measurement_disturbance_cov : ndarray The smoothed measurement disturbance covariance matrices at each time period. smoothed_state_disturbance_cov : ndarray The smoothed state disturbance covariance matrices at each time period. ) rscaled_smoothed_estimatorscaled_smoothed_estimator_covsmoothing_errorsmoothed_statesmoothed_state_covsmoothed_state_autocov smoothed_measurement_disturbancesmoothed_state_disturbance$smoothed_measurement_disturbance_covsmoothed_state_disturbance_covinnovations_transitionFc t|||jD]"}t||t ||d#d|_d|_d|_dS)a Update the results to match a given model Parameters ---------- model : Representation The model object from which to take the updated values. only_options : bool, optional If set to true, only the smoother and filter options are updated, and the state space representation is not updated. Default is False. Notes ----- This method is rarely required except for internal usage. N)r/r__smoother_optionsrTgetattr_smoothed_forecasts_smoothed_forecasts_error_smoothed_forecasts_error_cov)r6model only_optionsrUr>s r'r_z%SmootherResults.update_representation@st" %%e\:::* < > >   0   . / / /  " 4   2 3 3 3  & 8   6 7 7 7  $   J  (  60 J fT]++a/ - * *D(((##@@@$XtT::F)k)!#*@"DL+F+F+F"G"G"$&t!??&:$;$'$>$@FGIIINAc>??&:$;$'$;CG$D$F()+++NAc>??&:$;$'$E$G()+++~%'~tS-D&EANNN A:: E]]N3sU{Q6679B#~cEk1124A#~cEkAo5568AF4=))$111*5EWQZ%'F1eai(( A A519Ve^a%7785@&'' 2C8Q t}=>>D,.FD(.5(( )'E6' U"cAeaijjMAeffI,E&EF %'(( ) )6%)$D r)r cNd}|t|dkr||||f}|||jvr |j|Sd}|dkr| }d}|||td||}|dz}|d}||r|dkr||j|z dz}n|j}|i}|dks|dkrtd||krtd|dkr|jt d |dkrl||jdzkr^|j}||jdzkr8t j|d |df|jd d dffd j }n@|j ||}n/|dkr|j |s||jdzkr|dkrYt j |j |j |ft j z} t j| |j d d|dz ffd }n|j d |dz |dz f}|d dd}n|dkr,|j %|r#||jdzkr|j j ||}nR|r|||||}n6||||z ||z |} | dd d}| |d}n|dd d}| ||j|<|S)a Compute state vector autocovariances, conditional on the full dataset Computes: .. math:: Cov(\alpha_t - \hat \alpha_t, \alpha_{t - j} - \hat \alpha_{t - j}) where the `lag` argument gives the value for :math:`j`. Thus when the `lag` argument is positive, the autocovariance is between the current and previous periods, while if `lag` is negative the autocovariance is between the current and future periods. Parameters ---------- lag : int, optional The number of period to shift when computing the autocovariance. Default is 1. t : int, optional A specific period for which to compute and return the autocovariance. Cannot be used in combination with `start` or `end`. See the Returns section for details on how this parameter affects what is what is returned. start : int, optional The start of the interval (inclusive) of autocovariances to compute and return. Cannot be used in combination with the `t` argument. See the Returns section for details on how this parameter affects what is what is returned. Default is 0. end : int, optional The end of the interval (exclusive) autocovariances to compute and return. Note that since it is an exclusive endpoint, the returned autocovariances do not include the value at this index. Cannot be used in combination with the `t` argument. See the Returns section for details on how this parameter affects what is what is returned and what the default value is. extend_kwargs : dict, optional Keyword arguments containing updated state space system matrices for handling out-of-sample autocovariance computations in time-varying state space models. Returns ------- acov : ndarray Array of autocovariance matrices. If the argument `t` is not provided, then it is shaped `(k_states, k_states, n)`, while if `t` given then the third axis is dropped and the array is shaped `(k_states, k_states)`. The output under the default case differs somewhat based on the state space model and the sign of the lag. To see how these cases differ, denote the output at each time point as Cov(t, t-j). Then: - If `lag > 0` (and the model is either time-varying or time-invariant), then the returned array is shaped `(*, *, nobs)` and each entry [:, :, t] contains Cov(t, t-j). However, the model does not have enough information to compute autocovariances in the pre-sample period, so that we cannot compute Cov(1, 1-lag), Cov(2, 2-lag), ..., Cov(lag, 0). Thus the first `lag` entries have all values set to NaN. - If the model is time-invariant and `lag < -1` or if `lag` is 0 or -1, and the model is either time-invariant or time-varying, then the returned array is shaped `(*, *, nobs)` and each entry [:, :, t] contains Cov(t, t+j). Moreover, all entries are available (i.e. there are no NaNs). - If the model is time-varying and `lag < -1` and `extend_kwargs` is not provided, then the returned array is shaped `(*, *, nobs - lag + 1)`. - However, if the model is time-varying and `lag < -1`, then `extend_kwargs` can be provided with `lag - 1` additional matrices so that the returned array is shaped `(*, *, nobs)` as usual. More generally, the dimension of the last axis will be `start - end`. Notes ----- This method computes: .. math:: Cov(\alpha_t - \hat \alpha_t, \alpha_{t - j} - \hat \alpha_{t - j}) where the `lag` argument determines the autocovariance order :math:`j`, and `lag` is an integer (positive, zero, or negative). This method cannot compute values associated with time points prior to the sample, and so it returns a matrix of NaN values for these time points. For example, if `start=0` and `lag=2`, then assuming the output is assigned to the variable `acov`, we will have `acov[..., 0]` and `acov[..., 1]` as matrices filled with NaN values. Based only on the "current" results object (i.e. the Kalman smoother applied to the sample), there is not enough information to compute Cov(t, t+j) for the last `lag - 1` observations of the sample. However, the values can be computed for these time points using the transition equation of the state space representation, and so for time-invariant state space models we do compute these values. For time-varying models, this can also be done, but updated state space matrices for the out-of-sample time points must be provided via the `extend_kwargs` argument. See [1]_, Chapter 4.7, for all details about how these autocovariances are computed. The `t` and `start`/`end` parameters compute and return only the requested autocovariances. As a result, using these parameters is recommended to reduce the computational burden, particularly if the number of observations and/or the dimension of the state vector is large. References ---------- .. [1] Durbin, James, and Siem Jan Koopman. 2012. Time Series Analysis by State Space Methods: Second Edition. Oxford University Press. NrFT-Cannot specify both `t` and `start` or `end`.r /Negative `t`, `start`, or `end` is not allowed.`end` must be after `start`zdCannot return smoothed state covariances if those values have not been computed by Kalman smoothing..r r)r)lenrrXrr| RuntimeErrorrrrrr}rr8r transposer) r6lagtrrr cache_keyforward_autocovariancesrnansouts r'smoothed_state_autocovariancez-SmootherResults.smoothed_state_autocovarianceAsv  C $6$6!$;$;a,I  !TAAA7 B B#( 77$C&* # =e/3?LMM M =Ea%C =E ;& 377}7Li#o)i  M 199aNOO O ;;:;; ; !88/7 455 5 !88ty1},,*Ddi!m##~#uvv+&(@bcc(JKveCi( Qhh46B+C03ty1}0D0Dzzx s CDDrvM~46sHS1WH}EF23 #'8I3IJ>>!Q**DDQhh46B'C,/$)a-,?,?.0s;DD' .::=;BB 99cCi}:NN}}Q1-- =7DD>>!Q**D  >BD 0 ; r)Tc |||td||}|dz}| |jdz }||j}|dks|dkrtd||krtd|jtdd}|j|jkrt|d z|j|jkr |j|jkr|j|jkst|d z|jjD]{} | d kr t|| j d dk} t|| j d dk} | r| st|d | dz| s| rt|d| dz||?tj tj tj|t}||jkr|jjst#di} |jjD]_} | d kr t|| } t|| }| j d |j d kr| d|j d df| | <`|j|jj\}}|jd7||d| j}|||j}durdndur|jndkr |jzg}g}dkr>|D]:\}}|kr|||f#|||f;n|}t1|dkrt3|ddd}d}d}d}d}d}d}d}d}d} d}!t1|dkrt5|\}"}#tj|"d|"d dz}$|jddd|jf}tj||jt@<|j|$}|jj|$}||z }|j}%t3||j}&|&|jz }'|'dkr>tj!|%tj"|'|jftjzgd}%i}(|jjD]9} | d kr t|| } | j d dkr| dd|&f|(| <:|jj#|%fi|(})tHj%&|}*|)'|*|)(} t1|dkrtj|"d dz}$|"dz }+||+d}||+d}||+d}tj||},tSj*|)|,|$d|j+\}-}.}.|)j,j}/|/j ddkr|/|,}/||/dd|ddf}/|-dddd|f}-tj-|-d|/ddddddddfzd.dddd}t5fd|D\}"}#|"|"dz }0|.dddddddd|0|#f}||0|#f}||z}||0|#f}||0|#f}||d}|d}| jd||f}1|| jkrBt3|| j}2| jd7|2|d| }3tj!|1|3j/fd}1|1|z j}!||!d}!t1|dkr5t5|\}"}#|"|"dz }0||0|#f}||0|#f}||0|#f}t1|dkrt5|\}4}5tj0|4}6tj|4}7| |jd7|6|7dzd| jj}8n$| |6|7dzjj}8|jj|6|7dz}9|9|8z }:|4|6z };|9|;|5f}<|8|;|5f}=|:|;|5f}>|j,j ddkr|j,dd}nn||jkr'|j,d||f.ddd}n<|td|j1dkr |d}n|.ddd}|2|||| }?||dddd|f}|?dd|f}?||?z}@|@|>z}A||@d}@|Ad}An d}Ad}=d}d}@tgd7id |Ad!|d"|>d#|d$|d%|@d&|d'|=d(|<d)|d*|d+|d,|d-|d.|d/| d0|!d1|d2|d3|d4d5|d6|}B|BS)8u Compute the news and impacts associated with a data release Parameters ---------- previous : SmootherResults Prior results object relative to which to compute the news. This results object must have identical state space representation for the prior sample period so that the only difference is that this results object has updates to the observed data. t : int, optional A specific period for which to compute the news. Cannot be used in combination with `start` or `end`. start : int, optional The start of the interval (inclusive) of news to compute. Cannot be used in combination with the `t` argument. Default is the last period of the sample (`nobs - 1`). end : int, optional The end of the interval (exclusive) of news to compute. Note that since it is an exclusive endpoint, the returned news do not include the value at this index. Cannot be used in combination with the `t` argument. revisions_details_start : bool or int, optional The period at which to beging computing the detailed impacts of data revisions. Any revisions prior to this period will have their impacts grouped together. If a negative integer, interpreted as an offset from the end of the dataset. If set to True, detailed impacts are computed for all revisions, while if set to False, all revisions are grouped together. Default is False. Note that for large models, setting this to be near the beginning of the sample can cause this function to be slow. design : array, optional Design matrix for the period `t` in time-varying models. If this model has a time-varying design matrix, and the argument `t` is out of this model's sample, then a new design matrix for period `t` must be provided. Unused otherwise. state_index : array_like, optional An optional index specifying a subset of states to use when constructing the impacts of revisions and news. For example, if `state_index=[0, 1]` is passed, then only the impacts to the observed variables arising from the impacts to the first two states will be returned. Returns ------- news_results : SimpleNamespace News and impacts associated with a data release. Includes the following attributes: - `update_impacts`: update to forecasts of impacted variables from the news. It is equivalent to E[y^i | post] - E[y^i | revision], where y^i are the variables of interest. In [1]_, this is described as "revision" in equation (17). - `revision_detailed_impacts`: update to forecasts of variables impacted variables from data revisions. It is E[y^i | revision] - E[y^i | previous], and does not have a specific notation in [1]_, since there for simplicity they assume that there are no revisions. - `news`: the unexpected component of the updated data. Denoted I = y^u - E[y^u | previous], where y^u are the data points that were newly incorporated in a data release (but not including revisions to data points that already existed in the previous release). In [1]_, this is described as "news" in equation (17). - `revisions`: y^r(updated) - y^r(previous) for periods in which detailed impacts were computed - `revisions_all` : y^r(updated) - y^r(previous) for all revisions - `gain`: the gain matrix associated with the "Kalman-like" update from the news, E[y I'] E[I I']^{-1}. In [1]_, this can be found in the equation For E[y_{k,t_k} \mid I_{v+1}] in the middle of page 17. - `revision_weights` weights on observations for the smoothed signal - `update_forecasts`: forecasts of the updated periods used to construct the news, E[y^u | previous]. - `update_realized`: realizations of the updated periods used to construct the news, y^u. - `revised`: revised observations of the periods that were revised and for which detailed impacts were computed - `revised`: revised observations of the periods that were revised - `revised_prev`: previous observations of the periods that were revised and for which detailed impacts were computed - `revised_prev_all`: previous observations of the periods that were revised and for which detailed impacts were computed - `prev_impacted_forecasts`: previous forecast of the periods of interest, E[y^i | previous]. - `post_impacted_forecasts`: forecast of the periods of interest after taking into account both revisions and updates, E[y^i | post]. - `revision_results`: results object that updates the `previous` results to take into account data revisions. - `revision_results`: results object associated with the revisions - `revision_impacts`: total impacts from all revisions (both grouped and detailed) - `revisions_ix`: list of `(t, i)` positions of revisions in endog - `revisions_details`: list of `(t, i)` positions of revisions to endog for which details of impacts were computed - `revisions_grouped`: list of `(t, i)` positions of revisions to endog for which impacts were grouped - `revisions_details_start`: period in which revision details start to be computed - `updates_ix`: list of `(t, i)` positions of updates to endog - `state_index`: index of state variables used to compute impacts Notes ----- This method computes the effect of new data (e.g. from a new data release) on smoothed forecasts produced by a state space model, as described in [1]_. It also computes the effect of revised data on smoothed forecasts. References ---------- .. [1] Bańbura, Marta and Modugno, Michele. 2010. "Maximum likelihood estimation of factor models on data sets with arbitrary pattern of missing data." No 1189, Working Paper Series, European Central Bank. https://EconPapers.repec.org/RePEc:ecb:ecbwps:20101189. .. [2] Bańbura, Marta, and Michele Modugno. "Maximum likelihood estimation of factor models on datasets with arbitrary pattern of missing data." Journal of Applied Econometrics 29, no. 1 (2014): 133-160. Nrr rrrzECannot compute news without having applied the Kalman smoother first.zThis results object has %s and so it does not appear to by an extension of `previous`. Can only compute the news by comparing this results set to previous results objects.z"fewer observations than `previous`z0different state space dimensions than `previous`obsrz time-varying z while `previous` does notztime-invariant rLz[Cannot compute the impacts of news on periods outside of the sample in time-varying models..)rrTFr) compute_t compute_jcompute_prior_weightsr.Nr c,g|]}|dk|S)rr )r$srevisions_details_starts r' z(SmootherResults.news..|s3-8-8-8t666666r).rrzcModel has time-varying design matrix, so an updated time-varying matrix for period `t` is required.)rrrupdate_impactsrevision_detailed_impactsnews revisions revisions_allgainrevision_weightsupdate_forecastsupdate_realizedrevised revised_all revised_prevrevised_prev_allprev_impacted_forecastspost_impacted_forecastsrevision_resultsrevision_impacts revisions_ixrevisions_detailsrevisions_groupedr updates_ix state_indexr )4rXrr|r7r8r9rshapesr;rrr atleast_1dsortrinttime_invariantr diff_endogrBrpredictsmoothed_forecastsrrrziparanger2rrastypeboolrrcloner Initialization from_results initializerdr _compute_smoothed_state_weightsrdesignnansumr forecastsminndimsmoothed_state_gainr)Cr6previousrrrrr rerror_ssr%tvtv_prevrmatprev_matrrrrrrrr revised_endogrrrrrrrrrr revised_j revised_pr tmp_endogtmp_nobsoos_nobs clone_kwargsrev_modinitoffsetrsmoothed_state_weights_ZTix_jrevised_impact_forecasts predict_startpupdate_tupdate_kupdate_start_t update_end_trrealizedforecasts_errorix_trrupdate_forecasts_error state_gainobs_gainrrsC ` r'rzSmootherResults.news#s | =e/3?LMM M =Ea%C =IME ;)C 199aNOO O %<<:;; ;  " *788 8  9x} $ $X)++,, ,  000 !222 !222X)++,, ,:$))++ 9 9Ce||s##)"-1Bh,,226:G 9' 9 -8C-8-8-8"8999 9' 9 -8c-8-8-8"8999  "-C88899;;K ??4:##<#>@@M<>6M(*11$77 8'/)4K (~/ : )+;;M &,,..I3 ..H(-/H!||Nrx4<(@AABFJ, Lz(--// < <%<<dC((9R=1$$(+C(N(;L%*hn*9EE EEG!0==dCCD   t $ $ $&~~// $%%))I&=&/ma&799 19Q<?%fgg./8 )&''2 IeS11 99 .38>KKK-&1^%8A;??IB*AAA{AAA-.B.qqq!!![/@A+$&9*95D!!!T111,-.45$7$7$77@yAq!7L7L!"(+-8-8-8-8+-8-8-8(9$ 9!9Q</%..q!Q::111aaa;?-q{;!,E,E,E(!9 7!89: }#3A#6  |  q #& #5 Iyy|+D)$ /:M%dIo6K/i@  z??Q  !$j!1 HhVH--N6(++L',H,%(lQ.>%%#%%%7 -44($q(5***>q!QGG>$&@AAA[A%%#I.FF#--aA66F!55%S 6OOJ&111k 12';7  *H&(>>N}#A;!/!2!N# "O%) "H;%;%;%*>;% '@&? ;%(';% i;%(-;%;%".-#;%(.-);%.,O/;%6G7;%<$ =;%D&E;%J.-K;%N%<$;O;%R%<$;S;%V.-W;%Z.-[;%^&_;%d0/e;%j0/k;%n%<$;o;%r"zs;%v$ w;%z r)c \|||td||}|dz}| jdz }|j}i|dks|dkrtd||krtd||z }tfdfd}tj|jf}t ||D]} || || |z <tjf} t D] } | \} } t | dzD]}|\}}d | }d |}| |z |  }tj|| | dz|z|||dzj zx| | |f<| || f<| |krad | }| |kr| | |fxx|| |fz cc<| | |fxx|| |fz cc<| || fxx|| |fz cc< |tj | z}||d}|S) ae Cov(\tilde \alpha_{t}, I) Var(I, I)^{-1} where I is a vector of forecast errors associated with `update_indices`. Parameters ---------- updates_ix : list List of indices `(t, i)`, where `t` denotes a zero-indexed time location and `i` denotes a zero-indexed endog variable. Nrr rrrc$t|}|jddkre|jkr |d|f}nW|vs|jd|jz krtd|d|d|jz f}n|d}|S)Nrr .zModel has time-varying zP matrix, so an updated time-varying matrix for the extension period is required.r)rrrrX)whichrrrrr6s r'get_matz4SmootherResults.smoothed_state_gain..get_mat[s$&&Cy}q  ty==c1f+CC]22)%06r:a$)mKK(*65*6*6*6777(.sA M/ABCC&kJr)ctj j f}t D]U} |\}} ||z |}d|}||||dzjz|dd||dzf<V|S)Nrrrr r )rrr8rrr) rtmp1rt_ik_irZ_irr7 n_updatesr6rs r'get_cov_state_revisionzCSmootherResults.smoothed_state_gain..get_cov_state_revisionms8T]I677D9%% = =%a=S99C1M:CCgh,,#'#c#'k*:*<#<QQQ!a%Z  Kr)r r9r) rXrrrrr8rrsqueezerlinalginv)r6rrrrr n_periodsr?r:rtmp2rr;r<jt_jk_jr=Z_jrHrr7r>s`` ` @@r'rz#SmootherResults.smoothed_state_gain3s =e/3?LMM M =Ea%C =IME ;)C  M 199aNOO O %<<:;; ;%K  OO       $         xDM9=>>uc"" 8 8A44Q77DUOOxI.//y!! 2 2A!!}HC1q5\\ 2 2%a=Sgh,,gh,,99c S :GG*,*C!G $t+c#cAg+.>.@@++QT T!Q$Z#:: 3//AAvvQT aSk1 QT aSk1 QT aSk1 ' 2,bimmD))) =7D r)c |jB````D6:DHNNNN`=A48hhhhT% % % N11X111X111X1;K/3TDTDTDTDTDTDTDTDr)r,)rhnumpyrtypesr)statsmodels.tsa.statespace.representationr(statsmodels.tsa.statespace.kalman_filterrr statsmodels.tsa.statespace.toolsrrr statsmodels.tsa.statespacer r rirjrkrlrmrnrorqrprrrr,r r)r'rrs!!!!!!CCCCCCEEEEEEEEGGGGGGGGGG<<<<<<<<''*>>56 GGGGG\GGGT xDxDxDxDxDmxDxDxDxDxDr)