M/Ph[jddlZddlZddlmZmZddlmZ ddZ ddZ dZ Gdd Z Gd d Z dd Zdd ZddZdS)N)chi2norm)utilsTc|tj|d\}}nFtj||f}tj|d\}}|dt|}t|} |tj||| } ntj|||z| } |tj|| } ntj||| } |{tj| | z } tj||d} |tj| | } ntj| || } tj| | z } | | z } n&tj| ddd ddd } |r0tj| dk}| |} | |} ||}d | | tj z z }|d k}d ||<tj |}tj|}tj |}d||<|s|||| | fS|| | | z z}tj |d tj }| |tj z }tj|| | k| dkz<tj|}tj|}tj||dkz}||xx||zcc<tj||<nM| | | ztj z }tj|}tj|}||||| | fS) zX Calculate the survival function and its standard error for a single group. NTreturn_inverserweights minlengthr leftsidegؗҜ= 1 indicates which event occurred at time t. If status = 0, the subject was censored at time t. title : str Optional title used for plots and summary output. freq_weights : array_like Optional frequency weights exog : array_like Optional, if present used to account for violation of independent censoring. bw_factor : float Band-width multiplier for kernel-based estimation. Only used if exog is provided. dimred : bool If True, proportional hazards regression models are used to reduce exog to two columns by predicting overall events and censoring in two separate models. If False, exog is used directly for calculating kernel weights without dimension reduction. Attributes ---------- times : array_like The distinct times at which the incidence rates are estimated cinc : list of arrays cinc[k-1] contains the estimated cumulative incidence rates for outcome k=1,2,... cinc_se : list of arrays The standard errors for the values in `cinc`. Not available when exog and/or frequency weights are provided. Notes ----- When exog is provided, a local estimate of the cumulative incidence rate around each point is provided, and these are averaged to produce an estimate of the marginal cumulative incidence functions. The procedure is analogous to that described in Zeng (2004) for estimation of the marginal survival function. The approach removes bias resulting from dependent censoring when the censoring becomes independent conditioned on the columns of exog. References ---------- The Stata stcompet procedure: http://www.stata-journal.com/sjpdf.html?articlenum=st0059 Dinse, G. E. and M. G. Larson. 1986. A note on semi-Markov models for partially censored data. Biometrika 73: 379-386. Marubini, E. and M. G. Valsecchi. 1995. Analysing Survival Data from Clinical Trials and Observational Studies. Chichester, UK: John Wiley & Sons. D. Zeng (2004). Estimating marginal survival function by adjusting for dependent censoring using many covariates. Annals of Statistics 32:4. https://arxiv.org/pdf/math/0409180.pdf N?Tc, t||d|dtj|x}|_tj|x}|_|tj|x}|_|gddlm}tj|x}|_|j d} | dz|z fd} ||||| ||} | d|_ | d|_ dSt|||} | d|_ | d|_ | d|_ |sdn||_dS)Nr)_kernel_cumincidencerUUUUUUտcftj|dz dzz dSNr9rrrr?xkws r5z,CumIncidenceRight.__init__..-bfadURU]3377::r7r9)rRrasarrayr#r$rO_kernel_estimatesrWrPrNtimescincrJcinc_setitle) selfr#r$rgrOrP bw_factordimredrWnobskfuncr]r^s @r5__init__zCumIncidenceRight.__init__s9 4|T::::d+++ty!z&111  #/1z,/G/G GL4,   ? ? ? ? ? ?!z$// /D49:a=D)+B::::E$$T64 %+--A1DJ!DI F !$ = =aD t qT $/RR% r7)NNNrUT)__name__ __module__ __qualname____doc__rmr7r5rTrTs>HHT?C15000000r7rTcBeZdZdZ ddZddZdZdd Zd Zdd Z dS) SurvfuncRighta Estimation and inference for a survival function. The survival function S(t) = P(T > t) is the probability that an event time T is greater than t. This class currently only supports right censoring. Parameters ---------- time : array_like An array of times (censoring times or event times) status : array_like Status at the event time, status==1 is the 'event' (e.g. death, failure), meaning that the event occurs at the given value in `time`; status==0 indicates that censoring has occurred, meaning that the event occurs after the given value in `time`. entry : array_like, optional An array of entry times for handling left truncation (the subject is not in the risk set on or before the entry time) title : str Optional title used for plots and summary output. freq_weights : array_like Optional frequency weights exog : array_like Optional, if present used to account for violation of independent censoring. bw_factor : float Band-width multiplier for kernel-based estimation. Only used if exog is provided. Attributes ---------- surv_prob : array_like The estimated value of the survivor function at each time point in `surv_times`. surv_prob_se : array_like The standard errors for the values in `surv_prob`. Not available if exog is provided. surv_times : array_like The points where the survival function changes. n_risk : array_like The number of subjects at risk just before each time value in `surv_times`. Not available if exog is provided. n_events : array_like The number of events (e.g. deaths) that occur at each point in `surv_times`. Not available if exog is provided. Notes ----- If exog is None, the standard Kaplan-Meier estimator is used. If exog is not None, a local estimate of the marginal survival function around each point is constructed, and these are then averaged. This procedure gives an estimate of the marginal survival function that accounts for dependent censoring as long as the censoring becomes independent when conditioning on the covariates in exog. See Zeng et al. (2004) for details. References ---------- D. Zeng (2004). Estimating marginal survival function by adjusting for dependent censoring using many covariates. Annals of Statistics 32:4. https://arxiv.org/pdf/math/0409180.pdf NrUc t|||||tj|x}|_tj|x}|_|tj|x}|_|tj|x}|_|w|tdddlm }tj|x}|_ |j d} | dz|z fd} ||||| |} | d|_ | d|_ dSt||||} | d|_ | d|_| d|_ | d |_| d |_|sd n||_dS) Nz%exog and entry cannot both be presentr)_kernel_survfuncrrXcftj|dz dzz dSrZr[r\s r5r_z(SurvfuncRight.__init__..^r`r7)r r%r9ra)rRrrbr#r$rOr%rLrcrvrPrN surv_prob surv_timesr6 surv_prob_sen_riskn_eventsrg) rhr#r$r%rgrOrPrirvrkrlr]r^s @r5rmzSurvfuncRight.__init__Ks 4 d;;;:d+++ty!z&111  #/1z,/G/G GL4,  !#E!2!2 2EDJ    !HIII ; ; ; ; ; ;!z$// /D49:a=D)+B::::E  vtULIIAqTDNdDO F v|', . . .1aDA$d ! $/RR% r7c"t||S)a Plot the survival function. Examples -------- Change the line color: >>> import statsmodels.api as sm >>> data = sm.datasets.get_rdataset("flchain", "survival").data >>> df = data.loc[data.sex == "F", :] >>> sf = sm.SurvfuncRight(df["futime"], df["death"]) >>> fig = sf.plot() >>> ax = fig.get_axes()[0] >>> li = ax.get_lines() >>> li[0].set_color('purple') >>> li[1].set_color('purple') Do not show the censoring points: >>> fig = sf.plot() >>> ax = fig.get_axes()[0] >>> li = ax.get_lines() >>> li[1].set_visible(False) ) plot_survfunc)rhaxs r5plotzSurvfuncRight.plotns4T2&&&r7ctj|jd|z k}t|dkr tjS|j|dS)z Estimated quantile of a survival distribution. Parameters ---------- p : float The probability point at which the quantile is determined. Returns the estimated quantile. rr)rrrzrr r{)rhpr0s r5quantilezSurvfuncRight.quantilesE^DNQU2 3 3 r77a<<6Mr!u%%r7皙?cloglogctjd|dz z }|}|dkrd}d}nL|dkrd}d}n?|d krtj}d }n)|d krd }d }n|dkrd}d}nt d||j|d|z z }|||j|jzz}tjtj ||k}t|dkrtj tj fS|j |d} |dt|j dz kr tj } n|j |ddz} | | fS)a Returns a confidence interval for a survival quantile. Parameters ---------- p : float The probability point for which a confidence interval is determined. alpha : float The confidence interval has nominal coverage probability 1 - `alpha`. method : str Function to use for g-transformation, must be ... Returns ------- lb : float The lower confidence limit. ub : float The upper confidence limit. Notes ----- The confidence interval is obtained by inverting Z-tests. The limits of the confidence interval will always be observed event times. References ---------- The method is based on the approach used in SAS, documented here: http://support.sas.com/documentation/cdl/en/statug/68162/HTML/default/viewer.htm#statug_lifetest_details03.htm rr9rcPtjtj| SNrrr]s r5r_z+SurvfuncRight.quantile_ci..s"&"&)),,r7c6d|tj|zz S)Nrrrs r5r_z+SurvfuncRight.quantile_ci..srQ]3r7linearc|Srrrrs r5r_z+SurvfuncRight.quantile_ci..s!r7cdSNrrrrs r5r_z+SurvfuncRight.quantile_ci..sqr7rc d|z Srrrrs r5r_z+SurvfuncRight.quantile_ci..s q1ur7logitc6tj|d|z z Srrrs r5r_z+SurvfuncRight.quantile_ci..s"&a!e--r7cd|d|z zz Srrrrs r5r_z+SurvfuncRight.quantile_ci..sqAQK0r7asinsqrtcNtjtj|Sr)rarcsinr!rs r5r_z+SurvfuncRight.quantile_ci..s")BGAJJ//r7cfddtj|ztjd|z zz S)Nrr9)rr!rs r5r_z+SurvfuncRight.quantile_ci..s(qA NRWQU^^$CDr7zunknown methodrr)rppflowerrrrLrzr|rabsrr r{r) rhralphamethodtrggprimerr0lbubs r5 quantile_cizSurvfuncRight.quantile_cisFXa%!)m $ $ Y  ,,A33FF x   A [FF u__A$_FF w  --A00FF z ! !//ADDFF-.. . Adn  !a% ( ffT^$$t'889 ^BF1IIO , , r77a<<626> ! _RU # b6S))A- - -BBB!,B2v r7ctj|j}d|j_|j|d<|j|d<|j|d<|j|d<|S)z Return a summary of the estimated survival function. The summary is a dataframe containing the unique event times, estimated survival function values, and related quantities. )indexTimez Surv probz Surv prob SEz num at riskz num events) pd DataFramer{rnamerzr|r}r~)rhdfs r5summaryzSurvfuncRight.summarysV\ 0 0 0 .;!.> K==< r7hwrc|}|dkrd}t||dkrtd|}|jdz|jdzz }|j}|dkrit j|t j|jz}dd||zzz|z }t j|}|jd|z z} |j|z} n|d krd} | d||zzdt j|zz z} | t j|jd|jz z z} t j t j|j} t j | | z d t j } t j | dz} t j | | zt j t j dz } t j | dz} ntd | | fS) a Returns a simultaneous confidence band for the survival function. Parameters ---------- alpha : float `1 - alpha` is the desired simultaneous coverage probability for the confidence region. Currently alpha must be set to 0.05, giving 95% simultaneous intervals. method : str The method used to produce the simultaneous confidence band. Only the Hall-Wellner (hw) method is currently implemented. transform : str The used to produce the interval (note that the returned interval is on the survival probability scale regardless of which transform is used). Only `log` and `arcsin` are implemented. Returns ------- lcb : array_like The lower confidence limits corresponding to the points in `surv_times`. ucb : array_like The upper confidence limits corresponding to the points in `surv_times`. rz0only the Hall-Wellner (hw) method is implementedrzalpha must be set to 0.05r9rg_)Ǻ?rrrzUnknown transform)rrLr|rzr}rr!rrrrrsinpi)rhrr transformrQs2nnr2thetalcbucbrBfrIs r5simultaneous_cbzSurvfuncRight.simultaneous_cbs< T>>DCS// ! D==899 9OO%%   !DNA$5 5 [   GBKK"&"8"88Ea"r'k*U2EF5MME.1U7+C.%'CC ( " "A !b2g+!bgbkk/2 2A 1t~+=>?? ?A "'$.1122AAq"&))A&))Q,CAwa00A&))Q,CC011 1Cxr7)NNNNrUr)rr)rrr) rnrorprqrmrrrrrrrr7r5rtrtsAAF8<9;!0!0!0!0F''''8&&&*FFFFP";;;;;;r7rtc btj|}tj|}tj|}tj|}|t||||||fi|\}} nitj|}tj|} d\}} | D]9} || k} t|| || || |||fi|\} }|| z }| |z } :|tj| |}dtj|t|dz z }||fS)a9 Test for the equality of two survival distributions. Parameters ---------- time : array_like The event or censoring times. status : array_like The censoring status variable, status=1 indicates that the event occurred, status=0 indicates that the observation was censored. group : array_like Indicators of the two groups weight_type : str The following weight types are implemented: None (default) : logrank test fh : Fleming-Harrington, weights by S^(fh_p), requires exponent fh_p to be provided as keyword argument; the weights are derived from S defined at the previous event time, and the first weight is always 1. gb : Gehan-Breslow, weights by the number at risk tw : Tarone-Ware, weights by the square root of the number at risk strata : array_like Optional stratum indicators for a stratified test entry : array_like Entry times to handle left truncation. The subject is not in the risk set on or before the entry time. Returns ------- chisq : The chi-square (1 degree of freedom) distributed test statistic value pvalue : The p-value for the chi^2 test N)rr) rrbr _survdiffdotlinalgsolvercdfr)r#r$group weight_typestratar%kwargsgrobsvarstustr0obs1var1chisqpvalues r5survdiffr6sRN :d  D Z  F Ju  E 5  B ~T65+r"..&,..SSF##iS  BB,B"48VBZr#.EEE=CEEJD$ 4KC 4KCC GGBIOOC-- . .E %R++ +F &=r7c |tj|d\}}nDtjtj||fd\}}|dt|}d|D} |6t |D]&\} } ||| k} || } || | f| | <'gg}}t|}| D]\}}||k}tj|||}tj|||z|}|||atj||z }tj||d}tj||}tj||z }||z }n&tj|ddd ddd }||t|}t|}tj |d k}d}|| }|d kr|}n|d krtj |}n|d krd|vrd}t||d}d ||tjz z }tj|}tj|}tj|}||z}tj|d }d |d<ntdt|d z } tj|tj|dtjdddfz }!g}"g}#|d z }$tj|$dtj}$t-d | d zD]}|||!||zz }%|!d dddfjtjd | |d z |!|dddfz z}&|||z z|$z }'|&|'dddfz}(|||%z}%|dzdddf|(z}(|"|%| |#|(| dtj|"})tj|#}*|)|*fS)NTrrcg|]}|dfSrrr).0gr_is r5 z_survdiff..s ' ' 'DD$< ' ' 'r7r r rr rrgbtwfhfh_pz4weight_type type 'fh' requires specification of fh_pzweight_type not implementedg|=r9)axis)rrrr enumeraterr=rrr?rrr!rLrrrrrollvstackrrr<Teyeravelhstack)+r#r$rrrr%rutimesrtimestserB_r0entry1nriskobsvr+rentry0mkr-obr.r/nrr nrisk_totixr rQrr1dfsr groups_oe groups_var var_denomoevar_tensor_partvar_scalar_partrobs_vecvar_mats+ r5rrxs }4===2>4-#@#@26888#d)) $ ( 'B ' ' 'C bMM % %DAq2a5.B2YFeV_CFFb4E VB 6qj Kb 9 9 9 [b A A A B   ! q A_VV&AAAFVr222B2#BaBB1TTrT7##DDbD)B R d))CE I  A & &BG!'')) $  GG D gi((GG D V##L oo%&>DS9++BJ7777BB2BB$hGggq))GGAJJ:;; ; b''A+C %279eRV<>> import statsmodels.api as sm >>> from statsmodels.duration.survfunc import plot_survfunc >>> data = sm.datasets.get_rdataset("flchain", "survival").data >>> df = data.loc[data.sex == "F", :] >>> sf0 = sm.SurvfuncRight(df["futime"], df["death"]) >>> sf1 = sm.SurvfuncRight(3.0 * df["futime"], df["death"]) >>> fig = plot_survfunc([sf0, sf1]) >>> ax = fig.get_axes()[0] >>> ax.set_position([0.1, 0.1, 0.64, 0.8]) >>> ha, lb = ax.get_legend_handles_labels() >>> leg = fig.legend((ha[0], ha[1]), (lb[0], lb[1]), loc='center right') Change the line colors: >>> fig = plot_survfunc([sf0, sf1]) >>> ax = fig.get_axes()[0] >>> ax.set_position([0.1, 0.1, 0.64, 0.8]) >>> ha, lb = ax.get_legend_handles_labels() >>> ha[0].set_color('purple') >>> ha[1].set_color('orange') rrrrgzGroup %d-r9post)labellwwhere+ z points)mscolorrg)\(?)r create_mpl_axtypertrrrr{rzr;r#getattrstepr logical_notr$rrr get_colorset_ylim) survfuncsrfiggxsfr{rzmxtrlir0tijjr1s r5rrsP!"%%GC IaL!!]22222 K I&&))B^aS"-$899 NQC#677 "'ll B  cU(;<rsN00000000&&&&&&KO $L&L&L&L&^3333lDDD(d0d0d0d0d0d0d0d0Nllllllll^ <@????DhhhhVNNNNNNr7