M/Ph?dZddlZddlmZddlmZddlmZddl m Z ddl m Z ddl mZd$d Zd$d Zd$d Zd$d Zd%dZd&dZGdde ZGdde ZGddeZGddeZGddeZGddeZGddeZGdde Zd Zd!ZGd"d#ZdS)'aTreatment effect estimators follows largely Stata's teffects in Stata 13 manual Created on Tue Jun 9 22:45:23 2015 Author: Josef Perktold License: BSD-3 currently available ATE POM_0 POM_1 res_ipw 230.688598 3172.774059 3403.462658 res_aipw -230.989201 3403.355253 3172.366052 res_aipw_wls -227.195618 3403.250651 3176.055033 res_ra -239.639211 3403.242272 3163.603060 res_ipwra -229.967078 3403.335639 3173.368561 Lots of todos, just the beginning, but most effects are available but not standard errors, and no code structure that has a useful pattern see https://github.com/statsmodels/statsmodels/issues/2443 Note: script requires cattaneo2 data file from Stata 14, hardcoded file path could be loaded with webuse N) Substitution) block_diag)WLS)GMM)ContrastResults)indentTc||z }d|z d|z z }|r.||z}||z}||z }||z|z S)zmoment condition for average treatment effect This does not include a moment condition for potential outcome mean (POM). ?mean)paramsendogtindprobweightedw1w0wdiffs g/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/statsmodels/treatment/treatment_effects.py_mom_ater'sa +B t)T "B bggii bggii GE 5=6 !!c||z }d|z d|z z }|r.||z}||z}tj||z|dz ||z|dz fS)zamoment conditions for average treatment means (POM) moment conditions are POM0 and POM1 r r)r np column_stack)r rrrrrrs r_mom_atmr8sx +B t)T "B bggii bggii ?EBJ2EBJ4JK L LLrc||z d|z d|z z z}tjd|z |f}||||z zdddf|z}|S)z} moment condition for average treatment mean based on OLS dummy regression moment conditions are POM0 and POM1 rN)rrdot)r rrrrw treat_indmoms r_mom_olsr"Fsi t qv!d(++AT4 011I  f--- .4 89 DC Jrc||z d|z d|z z z}tj|tjt|f}||||z zdddf|z}|S)z moment condition for average treatment mean based on OLS dummy regression first moment is ATE second moment is POM0 (control) rN)rroneslenr)tmrrrrrr r!s r _mom_ols_ter'Usu t qv!d(++Arws4yy'9'9 :;;I  b))) *AAAtG 4y @C Jrc||n|j}|||}|j|z }|||z}|dddf|z}|SN)exogpredictr)r modelr*scalefittedresidr!s r _mom_olsexr0es[#44D ]]64 ( (F K& E   4.4 C Jrc,||z }d|z d|z z }| ||z}||z}|r.||z}||z}||z }||z||z||zfS)zKaverage treatment effect based on basic inverse propensity weighting. r r )rrrrprobtrrrs rate_ipwr3os +B t)T "B  e  e  bggii bggii GE EM   ! !EBJ#4#4#6#68I8I8K8K KKrc*eZdZdZdfd ZdZxZS)_TEGMMGeneric1YGMM class to get cov_params for treatment effects This combines moment conditions for the selection/treatment model and the outcome model to get the standard errors for the treatment effect that takes the first step estimation of the treatment model into account. this also matches standard errors of ATE and POM in Stata Fc t|dd||_||_||_|j||jj g|j_|rd|_ n#t|j jj |_ |r |j |_dSd|_dS)Nr)super__init__results_select mom_outcome exclude_tmoms__dict__updatedataxnamesk_selectr%r, param_namesr+r)selfr res_selectr;r<kwargs __class__s rr9z_TEGMMGeneric1.__init__s d+++(&* V$$$ 9  #!DI   CDMM 0 5 ABBDM  +3355DIIIDIIIrct||jz }|d|}||d}|jjj}|jr|j}n|jj|}g}|||j||d}| ||js4|jj |} | | tj |} | S)NT)r) r%rAr:r,rr<rr+r;append score_obsrr) rCr k_outcomer&p_tmrr moms_listmom_omom_tmomss rmomcondz_TEGMMGeneric1.momcondsKK$-/ JYJ ijj!"(.   ;9DD&,44T::D   TZtd KK! $'-77==E   U # # #y)) r)F__name__ __module__ __qualname____doc__r9rP __classcell__rFs@rr5r5sV2rr5c(eZdZdZfdZdZxZS)_TEGMMr6ct|dd||_||_|jjg|j_dSdSr))r8r9r:r;r?r@)rCrrDr;rFs rr9z_TEGMM.__init__sT d+++(& 9  #!DI    $ #rc.|dd}|dd}|jjj}|jj|}|||j||}t j||jj|f}|S)N)r:r,rr+r;rrrI)rCr r&rKrrmomtrOs rrPz_TEGMM.momconds BQBZabbz"(."(0066DJd;; $ 3 9 C CD I I KLL rrQrWs@rrYrYsQ"""""       rrYceZdZdZdZdS)_IPWGMMa GMM for aipw treatment effect and potential outcome uses unweighted outcome regression c|j}|j}|j}|jj}|j}|dd}|dd}t j|j |} t j | dd} | } |dkrd} n=|dvr| } n6|dvrd| z } n,t|t j r| } ntd|| z d|z d| z z z} | | | z} t j|t jt!|f} | || |z zdddf| z}|j|}t j||f}|S) Nr\{Gz?Gz?allrtreatedr untreatedcontrolr!incorrect option for effect_group)teffr: treatment model_poolr effect_grouprasarrayr,r+clip isinstancendarray ValueErrorrr$r%rrI)rCr rarDrrrnr&psprob_selrr2rr mm mom_selectrOs rrPz_IPWGMM.momconds Y& | #( BQBZ ABBZ:j.66r::;;78T400 5 EE ^ + +EE 8 8 8HEE  bj 1 1 BEE@AA A 4K1t8D1 1   JAOT273t99+=+=$>?? 59==,,,-qqq$w 7) C%//33 J/00 rNrRrSrTrUrPrrr_r_s- '''''rr_ceZdZdZdZdS)_AIPWGMMr`c|j}|j}|j}|d}tjt |dk}||}|jjjj d}|d}|d|dz} ||dzd|zdz} |d|zdzd} |jj} |j j} |j }|j }tj |j| }tj|dd}||}||}tj||f}| | |}t#| | }| | |}t#| | }t%||}|j}tj||||f}||z d|z z d|z z}||z |z |z}||z}||z}||z }||z }||z } tj|| f}|j| }!tj|!||!|fd}!tj|||!f}"|"S)Nrrr\rbrcr,axisrk treat_maskr:raranger%results0r,r*shaperesults1 exog_grouped endog_groupedror+rp concatenater0rrlrrI)#rCr rtrrDppommaskkpmp0p1rumod0mod1r*rrvprob0prob1rfitted0mom0fitted1mom1r;rcorrect0correct1tmean0tmean1aterwmpomrxrOs# rrPz_AIPWGMM.momconds| Y] & ayyV%%* K  " ( + AY AacE] AaC!AI  AaCEFF^{ {  :j.66r::;;78T400*%$~uen--,,r4(("D))),,r4(("D))) t,, |~tZK0$z2BCDDGOD1QX>GOt+d28#8#vo 2X} _b$Z ( ( %//33 ^Z %<%/ %;%=CDFFF K<== rNryrzrrr|r|s- 99999rr|ceZdZdZdZdS) _AIPWWLSGMMzc GMM for aipw-wls treatment effect and potential outcome uses weighted outcome regression c2|j}|j}|j}|d}tjt |dk}||}|jjjj d}|d}|d|dz} ||dzd|zdz} |dd} |jj} |j j} |j }|j }tj |j| }tj|dd}||}||}tj||f}d}d|z d|z z d|z d|z z dz z}d}||z ||z dz z}| | |}t#| | |dddfz}| | |}t#| | |dddfz}t%||}|j}tj||||f}||z d|z z d|z z}||z |z |z}||z}||z}||z } | |z }!||z }"tj|!|"f}!|j| }#tj|#||#|fd}#tj|!||#f}$|$S) Nrrr\MbP?+?r~rr)%rCr rtrrDrrrrrrrurrr*rrvrrrrww0ww1rrrrr;rrrrrrwrrxrOs% rrPz_AIPWWLSGMM.momcondXs Y] & ayyV%%* K  " ( + AY AacE] AaC!AI  BCC[{ {  :j.66r::;;78UE22*%$~uen--4xAI&1t8E *BQ*FGUldUlQ./,,r4(("D)))C4L8,,r4(("D)))C4L8 t,, |~tZK0$z2BCDDGOD1QX>GOt+d28#8#vo 2X} _b$Z ( ( %//33 ^Z %<%/ %;%=CDFFF K<== rNryrzrrrrRs2 AAAAArrceZdZdZdZdS)_RAGMMzqGMM for regression adjustment treatment effect and potential outcome uses unweighted outcome regression c|j}|d}tjt|dk}||}|jjjjd}|d}|d|dz}|| d}|jj} |jj} |j } | || } t|| } | || }t|| }t| |}|| z |z }| |z }tj ||f}|j.||j|jz dddfz}tj ||f}|S)Nrrr~)rkrrr%rr,r*rrrr+r0rrr2r )rCr rtrrrrrrrrr*rrrrmomoutrwrrOs rrPz_RAGMM.momconds\ YayyV%%* K  " ( + AY AacE] QBCC[{ { ,,r4(("D))),,r4(("D)))D$'' w  #~ _b$Z ( ( : ! 4:  1 11111d7; ;BF|,, rNryrzrrrrs- rrceZdZdZdZdS) _IPWRAGMMz: GMM for ipwra treatment effect and potential outcome c|j}|j}|j}|d}tjt |dk}||}|jjjj d}|d}|d|dz} ||dzd|zdz} |dd} |jj} |j j} |j }tj t |}d|| d<tj|j| }tj|dd}||}||}|j}|dkrdd|z z }d|z }d}nl|dvr%|d|z z }||z }||z }nC|d vr0d|z d|z z }d|z |z }d|z }||z}nt'd | | |}t)| | |dddfz}| | |}t)| | |dddfz}t+||}||z |z |z}||z |z}tj||f}|j| }tj||||fd }tj|||f}|S) Nrrr\rrrrdrergrjr~r)rkrr:rrr%rr,r*rrrzerossumror+rprnr rsr0rrrIr) rCr rtrrDrrrrrrrurrr*rrvrrrnrrsindrrrrr;rwrrxrOs rrPz_IPWRAGMM.momconds  Y] & ayyV%%* K  " ( + AY AacE] AaC!AI  BCC[{ { xJ((#$jnn    :j.66r::;;78UE22*%$( 5 a%iBUBDD ^ + +!e)$BB$))++%DD 8 8 8e)E *Be)u$BHD DIIKK DD@AA A,,r4(("D)))Bqqq$wK7,,r4(("D)))Bqqq$wK7 t,, "$ ,$$& _b$Z ( ( %//33 ^Z %<%/ %;%=CDFFF K<== rNryrzrrrrs2DDDDDrrc"eZdZdZfdZxZS)TreatmentEffectResultsa  Results class for treatment effect estimation Parameters ---------- teff : instance of TreatmentEffect class results_gmm : instance of GMMResults class method : string Method and estimator of treatment effect. kwds: dict Other keywords with additional information. Notes ----- This class is a subclass of ContrastResults and inherits methods like summary, summary_frame and conf_int. Attributes correspond to a z-test given by ``GMMResults.t_test``. c tt|j}t jd|f}d|d<d|d<ddg|dddf<||}|j|j||_ ||_ ||_ |j|gd|_ dS)Nr)rr)rrr\)ATEPOM0POM1) r8r9r%r rrt_testr=r>rk results_gmmmethodc_names) rCrkrrkwdsk_params constraintsttrFs rr9zTreatmentEffectResults.__init__#s {)**h8}--  D DV ArrE    , , R[))) &  T"""... r)rRrSrTrUr9rVrWs@rrrsB&/////////rraParameters ---------- return_results : bool If True, then a results instance is returned. If False, just ATE, POM0 and POM1 are returned. effect_group : {"all", 0, 1} ``effectgroup`` determines for which population the effects are estimated. If effect_group is "all", then sample average treatment effect and potential outcomes are returned If effect_group is 1 or "treated", then effects on treated are returned. If effect_group is 0, "treated" or "control", then effects on untreated, i.e. control group, are returned. disp : bool Indicates whether the scipy optimizer should display the optimization results Returns ------- TreatmentEffectsResults instance or tuple (ATE, POM0, POM1) aAParameters ---------- return_results : bool If True, then a results instance is returned. If False, just ATE, POM0 and POM1 are returned. disp : bool Indicates whether the scipy optimizer should display the optimization results Returns ------- TreatmentEffectsResults instance or tuple (ATE, POM0, POM1) c:eZdZdZddZeddZdd Zee e d  dd Z ee e d  ddZ ee e d  ddZee e d  ddZdS)TreatmentEffecta Estimate average treatment effect under conditional independence .. versionadded:: 0.14.0 This class estimates treatment effect and potential outcome using 5 different methods, ipw, ra, aipw, aipw-wls, ipw-ra. Standard errors and inference are based on the joint GMM representation of selection or treatment model, outcome model and effect functions. Parameters ---------- model : instance of a model class The model class should contain endog and exog for the outcome model. treatment : ndarray indicator array for observations with treatment (1) or without (0) results_select : results instance The results instance for the treatment or selection model. _cov_type : "HC0" Internal keyword. The keyword oes not affect GMMResults which always corresponds to HC0 standard errors. kwds : keyword arguments currently not used Notes ----- The outcome model is currently limited to a linear model based on OLS. Other outcome models, like Logit and Poisson, will become available in future. See `Treatment Effect notebook <../examples/notebooks/generated/treatment_effect.html>`__ for an overview. NHC0c |j|tj||_|dkx|_}| ||_||_||_ |j }|j }|j d|_ ||_|||||} | ||_|||||} | ||_tj| j | j fd|_tj| j | j fd|_dS)Nrrcov_typer)r=r>rrorlrr:r+ prob_selectrmrr*rnobs _cov_typerFfitrrrrr) rCr,rlr:rrrrr*rrs rr9zTreatmentEffect.__init__s8 T"""I..(1Q7*  %"0D -5577D  zKN "uj[14 3DEE)44 uZ0$z2BCC)44 NDIty+AJJJ^TZ,D1MMMrolsc t)z>create models from data not yet implemented )NotImplementedError)clsrr*rlr,rs r from_datazTreatmentEffect.from_datas "!rTrdFc$|jj}|j}|j}|dkrd}nC|dvr|}d}n:|dvrd|z }d}n.t |t jr|}d}ntdt|||d | }|s|St||j d|| } t j |dd |j j f} | | t jt| d d|dd} t!|| d| |} | S)aInverse Probability Weighted treatment effect estimation. Parameters ---------- return_results : bool If True, then a results instance is returned. If False, just ATE, POM0 and POM1 are returned. effect_group : {"all", 0, 1} ``effectgroup`` determines for which population the effects are estimated. If effect_group is "all", then sample average treatment effect and potential outcomes are returned. If effect_group is 1 or "treated", then effects on treated are returned. If effect_group is 0, "treated" or "control", then effects on untreated, i.e. control group, are returned. disp : bool Indicates whether the scipy optimizer should display the optimization results Returns ------- TreatmentEffectsResults instance or tuple (ATE, POM0, POM1) See Also -------- TreatmentEffectsResults rdNrerrgruserrjT)rr2rkrnr\nmmaxiterdisp start_params inv_weights optim_method optim_argsrIPWrrn)rmrrlrrqrrrrsr3r_r:rr reyer%r) rCreturn_resultsrnrrrrr2res_ipwgmmrres_gmmress ripwzTreatmentEffect.ipwsp:%~ 5 EE ^ + +ELL 8 8 8HELL  bj 1 1 B E!LL@AA A%td%HHH NeT0$T#/111~wrr{'+':'A'CDD ''|&(fS->->&?&?'+15t%D%D"# %T7E2>2>''' rz )params_returnsctjt|j}d||j d<|dkrd}nC|dvr|}d}n:|dvrd|z }d}n.t |tjr|}d}ntd|j}||| z }nd}|j ||z }|j ||z } |s| |z || fS|j j} t| |jd|| } tj| |z |g|j j|j jf} | | tjt| d d |d d } t+|| d| |}|S)z Regression Adjustment treatment effect estimation. %(params_returns)s See Also -------- TreatmentEffectsResults rNrdrergrrrj)rkr2rrrrrr)rrr%rlrrqrrrsrr rr+rrmrrr:rr rrr)rCrrnrrr2r*cwpom0pom1rmod_gmmrrrs rrtzTreatmentEffect.ras xDN++,,'(dn  "" " # #$ 5 EE ^ + +ELL 8 8 8HELL  bj 1 1 B E!LL@AA A   %**,,&BBB %%d++b06688 %%d++b06688 +$;d* *% 3T$&&&~ D[$  M M '"## ++<*,&\1B1B*C*C+/594)H)H&'    %T7E2>2>''' rcZ|j}|j}|j}|jj}|jjd||dkz z |z }|jj||dkz |z }|j | |z} |j | |z} | | z } |s| | | fS|jj } tj | | g} t| |jd|}tj| |jj|jj|jjf}||tjt)|dd|dd}t+||d |d }|S) z ATE and POM from double robust augmented inverse probability weighting %(params_returns)s See Also -------- TreatmentEffectsResults rrNrkrrrrrrdr)rrrlrmr*rr/rrr+r rrror|r:rr rrr%r)rCrrrrrr*rrrrrrp2_aipw mag_aipw1rrrs raipwzTreatmentEffect.aipw,sy~#M'1tDAI+>?DDFFMM'4 ?;@@BBTI&&t,,1133h>&&t,,1133h>vo '& &%*c6]++UD$7DIII ~  M $-"6   &'()) --%s<0011#'66  %T7E2>27''' rc|j}|j}|jj}|jj}|j}|j}||z ||z dz z} t||||| |} | d} | | } d|z d|z z d|z d|z z dz z} t||||| |}|d}| | }||_ | |_ |j d||dkz z |z }| j ||dkz |z }||z}| |z}||z }|s|||fStj||g}t%||jd|}tj||j| j|jjf}||tjt/|dd |d d }t1||d |d }|S)ah ATE and POM from double robust augmented inverse probability weighting. This uses weighted outcome regression, while `aipw` uses unweighted outcome regression. Option for effect on treated or on untreated is not available. %(params_returns)s See Also -------- TreatmentEffectsResults rweightsHC1rrNrrrrrrrdr)rrrmrr*rlrrrr+r results_ipwwls0results_ipwwls1r/rrrosqueezerr:rr rr%r)rCrrrrrr*rrrrresult1 mean1_ipw2rrresult0 mean0_ipw2rrrrr p2_aipw_wlsrrrrs raipw_wlszTreatmentEffect.aipw_wlsXsy%#~_ TkTD[1_-5$d:&6z?,,,((E(**__T**//11 4xAH%!d(q4x)@1)DE5*%tZK'8 {+---((E(**__T**//11 &&MQdai%89>>@@4GMT$!)_5::<27''' rc|j}|jj}|jj}|j}||}||} |dkrdd|z z } d| z } |} nN|dvr|d|z z } | | z } ||} d}n2|dvrd|z d|z z } d| z | z } ||} d}nt dt ||||| } | d }||  }t ||||| }|d }||  }|s||z ||fSt||j d || }tj ||z |g|j|jtj|j jf}||tjt#|d d |dd}t%||d||}|S)z ATE and POM from inverse probability weighted regression adjustment. %(params_returns)s See Also -------- TreatmentEffectsResults rdrrergrrjrrrNrrirrrr)rrmrr*rrsrrr+r rr:rrr rorr%r)rCrrnrrrr*rrrrrexogtrr mean0_ipwrarr mean1_ipwrarrrrs ripw_razTreatmentEffect.ipw_ras_ %#j[!Z  5 a%iBUBEE ^ + +!e)$BB$ELL 8 8 8e)E *Be)u$B*%ELL@AA A5*%tZK'8((E(**ooe,,1133 5$d:&6((E(**ooe,,1133  G,k;F FE4#64)5777~ ; & 4 N N Jt*1 2 2 ' ++%s<0011#'66 %T7E2>2>''' r)Nr)r)TrdF)TF)rRrSrTrUr9 classmethodrrrrdoc_params_returnsrtdoc_params_returns2rrrrzrrrr]sm""HNNNN>"""["DDDDL\(:G!D!DEEE999FE9v\(;W!E!EFFF)))GF)V\(;W!E!EFFFAAAGFAF\(:G!D!DEEEFFFFEFFFrr)T)NNN)TN) rUnumpyrstatsmodels.compat.pandasr scipy.linalgr#statsmodels.regression.linear_modelr"statsmodels.sandbox.regression.gmmrstatsmodels.stats.contrastrstatsmodels.tools.docstringrrrr"r'r0r3r5rYr_r|rrrrrrrrzrrr s:222222######333333222222666666......""""" M M M M         LLLL(99999S999xSB-----n---`?????~???DGGGGG.GGGT%%%%%^%%%PHHHHHHHHV#/#/#/#/#/_#/#/#/L0  FFFFFFFFFFr