Propensity Score Matching in Stata using teffects
For many years, the standard tool for propensity score matching in Stata has been the psmatch2 command, written by Edwin Leuven and Barbara Sianesi. However, Stata 13 introduced a new teffects command for estimating treatments effects in a variety of ways, including propensity score matching. We are still learning about the strengths and weaknesses of teffects relative to psmatch2, but we generally encourage people to use official Stata commands when they are available and appropriate. Official commands are thoroughly tested and backed by the reputation of Stata Corporation, and they tend to be more robust and flexible (and often better documented). One clear advantage of teffects over psmatch2 is that teffects can use by and Stata's factor and interaction syntax.
This article is primarily intended for people who have used psmatch2 and want to learn teffects. At this point the article is a work in progress and it will be updated as issues are identified.
An Example of Propensity Score Matching
Run the following command in Stata to load an example data set:
use http://ssc.wisc.edu/sscc/pubs/files/psm
It consists of four variables: a treatment indicator t, covariates x1 and x2, and an outcome y. This is constructed data, and the effect of the treatment is in fact a one unit increase in y. However, the probability of treatment is positively correlated with x1 and x2, and both x1 and x2 are positively correlated with y. Thus simply comparing the mean value of y for the treated and untreated groups badly overestimates the effect of treatment:
ttest y, by(t)
(Regressing y on t, x1, and x2 will give you a pretty good picture of the situation.)
The psmatch2 command will give you a better estimate of the treatment effect:
psmatch2 t x1 x2, out(y)
Probit regression Number of obs = 1000
LR chi2(2) = 185.92
Prob > chi2 = 0.0000
Log likelihood = -369.21074 Pseudo R2 = 0.2011
------------------------------------------------------------------------------
t | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
x1 | .4962478 .0577088 8.60 0.000 .3831407 .609355
x2 | .5686983 .0586157 9.70 0.000 .4538137 .683583
_cons | -1.147509 .0590417 -19.44 0.000 -1.263229 -1.03179
------------------------------------------------------------------------------
----------------------------------------------------------------------------------------
Variable Sample | Treated Controls Difference S.E. T-stat
----------------------------+-----------------------------------------------------------
y Unmatched | 2.37231243 -.334801231 2.70711366 .13826271 19.58
ATT | 2.37231243 1.05004054 1.3222719 .203464124 6.50
----------------------------+-----------------------------------------------------------
Note: S.E. does not take into account that the propensity score is estimated.
The teffects Command
You can carry out the same estimation with teffects. The basic syntax of the teffects command when used for propensity score matching is:
teffects psmatch (outcome) (treatment covariates)
In this case the basic command would be:
teffects psmatch (y) (t x1 x2)
However, the default behavior of teffects is not the same as psmatch2 so we'll need to use some options to get the same results. First, psmatch2 by default reports the average treatment effect on the treated (which it refers to as ATT). The teffects command by default reports the average treatment effect (ATE) but will calculate the average treatment effect on the treated (which it refers to as ATET) if given the atet option. Second, psmatch2 by default uses a probit model for the probability of treatment. The teffects command uses a logit model by default, but will use probit if the probit option is applied to the treatment equation. So to run the same model using teffects type:
teffects psmatch (y) (t x1 x2, probit), atet
Treatment-effects estimation Number of obs = 1000
Estimator : propensity-score matching Matches: requested = 1
Outcome model : matching min = 1
Treatment model: probit max = 1
------------------------------------------------------------------------------
| AI Robust
y | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATET |
t |
(1 vs 0) | 1.322272 .1222199 10.82 0.000 1.082725 1.561819
------------------------------------------------------------------------------
The average treatment effect on the treated is identical, other than being rounded at a different place. But note that teffects reports a very different standard error (we'll discuss why that is shortly), plus a Z-statistic, p-value, and 95% confidence interval rather than just a T-statistic.
Running teffects with the default options gives the following:
teffects psmatch (y) (t x1 x2)
Treatment-effects estimation Number of obs = 1000
Estimator : propensity-score matching Matches: requested = 1
Outcome model : matching min = 1
Treatment model: logit max = 1
------------------------------------------------------------------------------
| AI Robust
y | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATE |
t |
(1 vs 0) | 1.317825 .1384469 9.52 0.000 1.046474 1.589176
------------------------------------------------------------------------------
This is equivalent to:
psmatch2 t x1 x2, out(y) logit ate
Logistic regression Number of obs = 1000
LR chi2(2) = 187.13
Prob > chi2 = 0.0000
Log likelihood = -368.60971 Pseudo R2 = 0.2024
------------------------------------------------------------------------------
t | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
x1 | .920473 .1081765 8.51 0.000 .7084509 1.132495
x2 | 1.008989 .1068032 9.45 0.000 .7996586 1.218319
_cons | -2.006834 .1163029 -17.26 0.000 -2.234783 -1.778884
------------------------------------------------------------------------------
----------------------------------------------------------------------------------------
Variable Sample | Treated Controls Difference S.E. T-stat
----------------------------+-----------------------------------------------------------
y Unmatched | 2.37231243 -.334801231 2.70711366 .13826271 19.58
ATT | 2.37231243 1.18607805 1.18623438 .204724198 5.79
ATU |-.334801231 1.01074393 1.34554516 . .
ATE | 1.31782508 . .
----------------------------+-----------------------------------------------------------
Note: S.E. does not take into account that the propensity score is estimated.
The ATE from this model is very similar to the ATT/ATET from the previous model. But note that psmatch2 is reporting a fairly different ATT in this model. The teffects command reports the same ATET if asked:
teffects psmatch (y) (t x1 x2), atet
Treatment-effects estimation Number of obs = 1000
Estimator : propensity-score matching Matches: requested = 1
Outcome model : matching min = 1
Treatment model: logit max = 1
------------------------------------------------------------------------------
| AI Robust
y | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATET |
t |
(1 vs 0) | 1.186234 .118919 9.98 0.000 .9531573 1.419311
------------------------------------------------------------------------------
This data set was constructed using a logit model for the probability of treatment, so it's not surprising that an estimate calculated using a logit model is closer to the actual treatment effect (which is 1).
Standard Errors
The output of psmatch2 includes the following caveat:
Note: S.E. does not take into account that the propensity score is estimated.
A recent paper by Abadie and Imbens (2012. Matching on the estimated propensity score. Harvard University and National Bureau of Economic Research) established how to take into account that propensity scores are estimated, and teffects psmatch relies on their work. Interestingly, the adjustment for ATE is always negative, leading to smaller standard errors: matching based on estimated propensity scores turns out to be more efficient than matching based on true propensity scores. However, for ATET the adjustment can be positive or negative, so the standard errors reported by psmatch2 may be too large or to small.
Handling Ties
Thus far we've used psmatch2 and teffects to do simple nearest-neighbor matching with one neighbor (and no caliper). However, this raises the question of what to do when two observations have the same propensity score and are thus tied for "nearest neighbor." Ties are common if the covariates in the treatment model are categorical or even integers.
The psmatch2 command by default matches with one of the tied observations, but with the ties option it matches with all tied observations. The teffects command always matches with all ties. If your data set has multiple observations with the same propensity score, you won't get exactly the same results from teffects as you were getting from psmatch2 unless you go back and add the ties option to your psmatch2 commands. (At this time we are not aware of any clear guidance as to whether it is better to match with ties or not; we will update this article if we find any.)
Matching With Multiple Neighbors
By default teffects psmatch matches each observation with one other observation. You can change this with the nneighbor() (or just nn()) option. For example, you could match each observation with its three nearest neighbors with:
teffects psmatch (y) (t x1 x2), nn(3)
In this data set increasing the number of neighbors brings the ATE somewhat closer to the constructed treatment effect of 1 (1.26 with three neighbors vs. 1.32 with one).
Postestimation
By default teffects psmatch does not add any new variables to the data set. However, there are a variety of useful variables that can be created with options and post-estimation predict commands. The following table lists the 1st and 15th observations of the example data set after some of these variables have been created. We'll refer to it as we explain the commands that created the new variables. (Reviewing these variables can also be a good way to make sure you understand exactly how propensity score matching works.)
+-----------------------------------------------------------------------------------------------------+
| x1 x2 t y match1 ps0 ps1 y0 y1 te |
|-----------------------------------------------------------------------------------------------------|
1. | -.0043712 .9755321 0 1.468089 15 .7362477 .2637523 1.468089 3.041705 1.573615 |
15. | .2281743 .7584399 1 3.041705 121 .7372159 .2627841 1.877903 3.041705 1.163802 |
+-----------------------------------------------------------------------------------------------------+
Start with a clean slate by typing:
clear
use http://ssc.wisc.edu/sscc/pubs/files/psm
The gen() option tells teffects psmatch to create a new variable (or variables). For each observation, this new variable will contain the number of the observation that observation was matched with. If there are ties or you told teffects psmatch to use multiple neighbors, then gen() will need to create multiple variables. Thus you supply the stem of the variable name, and teffects psmatch will add suffixes as needed.
teffects psmatch (y) (t x1 x2), gen(match)
In this case each observation is only matched with one other, so gen(match) only creates match1. Referring to the example output, the match of observation 1 is observation 15 (which is why those two are listed).
Note that these observation numbers are only valid in the current sort order, so make sure you can recreate that order if needed. If necessary, run:
gen obsNum=_n
and then:
sort obsNum
will restore the current sort order.
The predict command with the ps option creates two variables containing the propensity scores, or that observation's predicted probability of being in either the control group or the treated group:
predict ps0 ps1, ps
Here ps0 is the predicted probability of being in the control group (t=0) and ps1 is the predicted probability of being in the treated group (t=1). Observations 1 and 15 were matched because their propensity scores are very similar.
The po option creates variables containing the potential outcomes for each observation:
predict y0 y1, po
Because observation 1 is in the treated group, y1 contains its observed value of y. y0 is what y would have been if observation 1 had been in the control group instead. Under the propensity score matching estimator, y0 is the observed value of y for observation 1's match, observation 15. Observation 15 is in the control group, so its value for y0 is its observed value of y while its value for y1 is the observed value of y for its match, observation 121.
Running the predict command with no options gives the treatment effect itself:
predict te
The treatment effect is simply the difference between y1 and y0. You could calculate the ATE yourself (but emphatically not its standard error) with:
sum te
and the ATET with:
sum te if t
Other Methods of Estimating Treatment Effects
While propensity score matching is the most common method of estimating treatments effects at the SSCC, teffects also implements Regression Adjustment (teffects ra), Inverse Probability Weighting (teffects ipw), Augmented Inverse Probability Weighting (teffects aipw), Inverse Probability Weighted Regression Adjustment (teffects ipwra), and Nearest Neighbor Matching (teffects nnmatch). The syntax is similar, though it varies whether you need to specify variables for the outcome model, the treatment model, or both:
teffects ra (y x1 x2) (t)
teffects ipw (y) (t x1 x2)
teffects aipw (y x1 x2) (t x1 x2)
teffects ipwra (y x1 x2) (t x1 x2)
teffects nnmatch (y x1 x2) (t)
For this data set, regression adjustment comes closest to the correct treatment effect of 1; it is the only one that comes closer than simply regressing y on t, x1, and x2.
Complete Example Code
The following is the complete code for the examples in this article.
clear all
use http://ssc.wisc.edu/sscc/pubs/files/psm
ttest y, by(t)
reg y t x1 x2
psmatch2 t x1 x2, out(y)
teffects psmatch (y) (t x1 x2, probit), atet
teffects psmatch (y) (t x1 x2)
psmatch2 t x1 x2, out(y) logit ate
teffects psmatch (y) (t x1 x2), atet
teffects psmatch (y) (t x1 x2), nn(3)
teffects ra (y x1 x2) (t)
teffects ipw (y) (t x1 x2)
teffects aipw (y x1 x2) (t x1 x2)
teffects ipwra (y x1 x2) (t x1 x2)
teffects nnmatch (y x1 x2) (t)
clear
use http://ssc.wisc.edu/sscc/pubs/files/psm
teffects psmatch (y) (t x1 x2), gen(match)
predict ps0 ps1, ps
predict y0 y1, po
predict te
l if _n==1 | _n==15
Last Revised: 8/9/2013
RSS |