6 Propensity Score Methods
6.1 Motivation: The Curse of Dimensionality
Chapter 5 established that under consistency, conditional exchangeability, positivity, and integrability, the ATE is identified by the back-door adjustment formula, and developed three estimation strategies: regression adjustment, standardization, and stratification. All three require, in practice, that we condition on a covariate vector \(X\). When \(X\) is low-dimensional, this is feasible. When \(X\) has many components, it is not.
The problem. Stratification requires cells \(\{X = x\}\) to contain both treated and control units. With \(p\) binary covariates, there are \(2^p\) cells. With \(p = 10\), that is 1024 cells — far more than most datasets can support. Regression adjustment avoids the cell-count problem but requires a correctly specified model for \(\E[Y \mid T, X]\), which becomes harder to specify reliably as \(p\) grows.
The solution. Rosenbaum and Rubin (1983) showed that it is sufficient to condition on a single scalar function of \(X\): the propensity score \(\pi(X) = P(T{=}1 \mid X)\). The propensity score reduces the adjustment dimension without changing the identified estimand, provided unconfoundedness and positivity hold. Identification power is preserved; finite-sample efficiency, covariate balance, and robustness to model misspecification, however, depend on how well the score is estimated and how it is used in the downstream estimator.
6.2 The Propensity Score and Its Balancing Properties
The central insight of Rosenbaum and Rubin (1983) is that the \(p\)-dimensional adjustment problem of Chapter 5 reduces to a one-dimensional one. The right abstraction is the balancing score: any function of \(X\) that renders treatment assignment conditionally independent of the covariates. The propensity score is the canonical — and coarsest — such function.
6.2.1 Balancing Scores and the Propensity Score
The full covariate vector \(X\) is trivially a balancing score: \(T \indep X \mid X\) holds because, conditional on \(X\), the conditional distribution of \(X\) is degenerate and therefore cannot depend on treatment status. The propensity score is the coarsest balancing score (Lemma 6.1 below), and therefore provides the greatest dimension reduction without sacrificing identification.
In a randomized experiment, \(\pi(X)\) is a known constant for all units — equal to \(0.5\) under balanced Bernoulli assignment, and, marginally for each unit, under complete randomization with exactly \(n/2\) treated, where the assignment indicators are dependent because the treated count is fixed. In an observational study, \(\pi(X)\) varies with \(X\) and must be estimated from data.
Connection to strong ignorability. Under strong ignorability \((Y(0),Y(1)) \indep T \mid X\), the potential outcomes carry no further information about treatment assignment once \(X\) is given, so the treatment assignment mechanism satisfies \[P\bigl(T{=}1 \mid X,\, Y(0),\, Y(1)\bigr) = P(T{=}1 \mid X) = \pi(X).\] This equality is a consequence of the ignorability assumption, not part of the definition of \(\pi(X)\). What unconfoundedness rules out is precisely any residual dependence of treatment on the potential outcomes after conditioning on \(X\).
Theorem 6.1 (The Propensity Score Is a Balancing Score (Rosenbaum and Rubin 1983)) \(T \indep X \mid \pi(X)\).
Proof. It suffices to show \(P(T{=}1 \mid X, \pi(X)) = P(T{=}1 \mid \pi(X))\). Since \(\pi(X)\) is a deterministic function of \(X\), conditioning on both \(X\) and \(\pi(X)\) is the same as conditioning on \(X\) alone, so \(P(T{=}1 \mid X, \pi(X)) = P(T{=}1 \mid X) = \pi(X)\). For the right-hand side, the law of iterated expectations gives \[P\bigl(T{=}1 \mid \pi(X)\bigr) = \E\bigl[P(T{=}1 \mid X) \,\big|\, \pi(X)\bigr] = \E\bigl[\pi(X) \,\big|\, \pi(X)\bigr] = \pi(X),\] where the last equality holds because \(\pi(X)\) is measurable with respect to itself. Hence both sides equal \(\pi(X)\). \(\square\)
6.2.2 The Propensity Score Theorem
The balancing property has a fundamental identification consequence: unconfoundedness at \(X\) implies unconfoundedness at any balancing score of \(X\), and in particular at the scalar \(\pi(X)\). Before stating that result we record a structural fact about balancing scores: among all of them, the propensity score is the coarsest.
Lemma 6.1 (The Propensity Score Is the Coarsest Balancing Score (Rosenbaum and Rubin 1983)) If \(b(X)\) is any balancing score — that is, \(T \indep X \mid b(X)\) — then \(\pi(X)\) is a function of \(b(X)\): \(\pi(X) = g(b(X))\) for some measurable function \(g\).
Proof. Since \(b(X)\) is a balancing score, \(P(T{=}1 \mid X,\, b(X)) = P(T{=}1 \mid b(X))\). But \(b(X)\) is a function of \(X\), so conditioning on \(b(X)\) is already implicit in conditioning on \(X\): \[\pi(X) = P(T{=}1 \mid X) = P\bigl(T{=}1 \mid X,\, b(X)\bigr) = P\bigl(T{=}1 \mid b(X)\bigr).\] Hence \(\pi(X)\) is a measurable function of \(b(X)\) alone. \(\square\)
The lemma is the precise statement of the Rosenbaum–Rubin coarsest-balancing-score property: the propensity score is a measurable function of every balancing score, or equivalently \(\sigma\{\pi(X)\} \subseteq \sigma\{b(X)\}\). Every balancing score is therefore at least as fine as \(\pi(X)\), which is what makes \(\pi(X)\) the coarsest balancing score and the maximally parsimonious adjustment variable. The reverse factorization need not hold: a general balancing score — \(X\) itself, for example — need not be a function of \(\pi(X)\). (By analogy with classical sufficiency, \(\pi(X)\) plays the role of a minimal sufficient reduction of \(X\) for the treatment indicator, although no parametric model is involved here.)
Theorem 6.2 (Propensity Score Theorem (Rosenbaum and Rubin 1983)) Suppose joint conditional exchangeability holds: \((Y(0), Y(1)) \indep T \mid X\). Then for any balancing score \(b(X)\), \[\bigl(Y(0),\, Y(1)\bigr) \;\indep\; T \;\Big|\; b(X);\] no positivity is needed for this conditional-independence transfer. If, in addition, \(0 < \pi(X) < 1\) almost surely, then \(0 < P(T{=}1 \mid b(X)) < 1\) almost surely as well, so strong ignorability given \(X\) implies strong ignorability given \(b(X)\), and adjustment for \(b(X)\) identifies the potential-outcome means. In particular, taking \(b(X) = \pi(X)\) gives strong ignorability with the scalar propensity score in place of the full covariate vector.
6.3 Constructing the Propensity Score
The true propensity score is unknown in observational studies and must be estimated from the data. Its practical construction, however, requires more than choosing a binary-response algorithm. Three logically distinct decisions must be kept separate. First, causal reasoning determines the pretreatment adjustment set \(X\): the set must be sufficient to support conditional exchangeability, for example by blocking every back-door path from \(T\) to \(Y\) (Chapters 3 and 5). Second, statistical modeling determines how the treatment probability depends on the chosen covariates. Third, design diagnostics determine whether the fitted score actually produces adequate balance and overlap for the chosen estimand. A good predictive model for treatment does not automatically produce a good causal design, and no statistical treatment model can repair an invalid adjustment set chosen from an incorrect causal graph.
6.3.1 Covariate Selection: What Belongs in \(X\)?
The propensity score inherits its causal validity from the adjustment set on which it is conditioned. The primary rule is therefore graphical and scientific rather than predictive: \(X\) should be a pretreatment set sufficient to block all back-door paths from \(T\) to \(Y\). The causal assumption is \(Y(t) \indep T \mid X\); estimating \(\pi(X)\) cannot make this assumption true if it was false for the chosen \(X\).
Within a causally valid pretreatment set, candidate variables need not play identical roles. A useful first taxonomy classifies them by whether they predict treatment, outcome, both, or neither (Brookhart et al. 2006; VanderWeele 2019):
| Variable type | Predicts \(T\)? | Predicts \(Y\)? | Typical recommendation |
|---|---|---|---|
| Confounder | yes | yes | include |
| Outcome predictor only | no or weakly | yes | usually include |
| Treatment predictor only | yes | no | include only with caution |
| Predictor of neither | no | no | usually unnecessary |
Variables related to both treatment and outcome are central confounders and should ordinarily be included: they are needed to block back-door paths. Pretreatment variables that strongly predict the outcome may be useful even when they predict treatment only weakly, because they can improve precision and make residual imbalance less consequential. Variables that strongly predict treatment but are unrelated to the potential outcomes require more caution: although they improve prediction of \(T\), they can push estimated scores toward zero or one, reduce overlap and effective sample size, and inflate the variance of weighted estimators without removing appreciable confounding bias; moreover, conditioning on such a near-instrument can amplify the bias arising from any residual unmeasured confounding. Pure noise variables mainly add estimation variability and may worsen finite-sample balance.
These are tendencies rather than universal rules, and they do not justify automatic variable deletion. Whether a variable is truly unrelated to the potential outcomes is a scientific judgment subject to uncertainty, and plausible pretreatment confounders should generally be retained even when their treatment-model coefficients are small or statistically insignificant.
Two exclusions, by contrast, are categorical when the target is a total effect. Post-treatment variables should not be conditioned on: doing so can block part of the causal effect, induce selection bias, and change the estimand. Pretreatment colliders can be equally damaging despite being measured before treatment: in the M-structure of panel (b) below, the variable \(C\) is pretreatment, yet conditioning on it opens the noncausal path \(T \leftarrow U_1 \rightarrow C \leftarrow U_2 \rightarrow Y\) that is blocked when \(C\) is left alone (Chapter 3). “Include every available variable” is therefore not a safe default.
6.3.2 Parametric Estimation by Logistic Regression
The classical approach models \[\log\frac{\pi(X)}{1 - \pi(X)} = \beta_0 + \sum_{j=1}^{p} \beta_j X_j + \sum_{j=1}^{p} \gamma_j\, h_j(X_j) + \sum_{j<k} \delta_{jk}\, X_j X_k,\] where the \(h_j\) are optional nonlinear transformations (polynomials, splines) and the cross-product terms capture interactions among covariates, and estimates the coefficients by maximum likelihood. Linearity on the logit scale in the raw covariates is an assumption, not a default truth: nonlinear terms and covariate-by-covariate interactions are often needed to achieve balance. (Treatment-by-covariate interactions have no place in this model — treatment is the response here.) Terms should be added based on causal knowledge and the balance diagnostics of Section 6.3.5, not predictive fit alone.
Variable selection by treatment-model \(p\)-values is generally inappropriate. A covariate may be an important confounder or outcome predictor even when its coefficient in the treatment model is small or statistically insignificant, and removing it can worsen causal balance without materially changing treatment-prediction accuracy. Backward selection driven by significance serves sparse prediction, which is not the objective of a propensity model.
6.3.3 Flexible and Machine-Learning Estimators
When \(X\) is high-dimensional or when the true propensity score is non-linear, machine learning methods — gradient boosted trees, random forests, regularized logistic regression — can estimate \(\pi(X)\) more flexibly, reducing misspecification of the treatment model without fully prespecifying nonlinearities and interactions. Predictive accuracy alone, however, is not the relevant success criterion: hyperparameters tuned solely to classification loss (log loss, AUC) may improve prediction precisely by producing more extreme probability estimates, which destabilizes the resulting weights.
A related issue is overfitting: an estimator that nearly perfectly separates treated and control units in-sample produces fitted scores near one for treated units and near zero for controls, mimicking an overlap violation that need not exist in the population. Cross-fitting (Chapter 12) removes the same-sample component of this problem by evaluating each unit’s score on a fold not used to fit it, but it does not create overlap and does not prevent extreme out-of-fold predictions: a genuinely near-deterministic treatment rule yields extreme scores on held-out data as well. Regularization, overlap diagnostics, and — when scientifically justified — trimming or re-targeting the estimand remain separate requirements. Whatever the learner, the fitted design must ultimately be evaluated through the balance and overlap it delivers; no machine-learning method eliminates the need for causal covariate selection or design diagnostics.
6.3.4 The Goal Is Balance, Not Treatment Prediction
In ordinary classification, a model is rewarded for separating the two classes. In propensity-score analysis, strong separation signals trouble: fitted scores near one for treated units and near zero for controls mean little common support, extreme inverse-probability weights, and an estimator whose value depends on a handful of observations. A high AUC may therefore indicate a difficult causal comparison rather than a successful design. Predictive criteria are useful for model fitting, but they are not sufficient criteria for choosing a propensity-score design.
The estimated score, moreover, is a design device, not the final diagnostic object. Conditioning on the true score balances \(X\) in distribution (Theorem 6.1), but matching or weighting approximately on an estimated score does not guarantee adequate finite-sample balance. After matching or weighting, balance must therefore be assessed on the original covariates and, when scientifically relevant, on nonlinear functions and interactions of them; similar distributions of \(\hat\pi(X)\) across arms do not by themselves establish that the underlying covariates are balanced.
6.3.5 Balance and Overlap Diagnostics
The standard scalar diagnostic for covariate balance is the standardized mean difference (SMD). For covariate \(X_j\), let \(\bar X_{1j}\) and \(\bar X_{0j}\) denote the treated and control sample means and \(s_{1j}^2\), \(s_{0j}^2\) the corresponding sample variances. Then \[\mathrm{SMD}(X_j) = \frac{\bar X_{1j} - \bar X_{0j}}{\sqrt{\bigl(s_{1j}^2 + s_{0j}^2\bigr)/2}};\] for a binary covariate, the numerator is the difference in proportions, with the analogous pooled scale in the denominator. Dividing by the pooled scale makes the diagnostic free of the units of \(X_j\) and — unlike a \(t\)-statistic — insensitive to the sample size, so it measures the magnitude of imbalance rather than its statistical detectability. A common informal benchmark treats \(|\mathrm{SMD}| \le 0.1\) as adequate balance; this is a heuristic, not a test, and whether residual imbalance is consequential depends on how strongly the covariate predicts the outcome, on the sample size, and on the estimand. For weighted or matched analyses, the arm means are replaced by their design-weighted counterparts (with reused control units counted with their matching multiplicities); the denominator, however, should be held fixed at the pre-adjustment pooled scale, so that before-and-after values are directly comparable.
Balance of means alone can be inadequate when the outcome depends nonlinearly on the covariates. Useful supplements include variance ratios, quantile differences, empirical-distribution plots, and SMDs computed on squares, interactions, and other scientifically relevant transformations.
Balance and overlap are distinct, and both must be checked: a design can show good mean balance after weighting while relying on a few extreme weights, and raw overlap can look reasonable while nonlinear imbalance remains. Overlap diagnostics start with plots of the estimated score distributions in the two arms on a common scale. Which comparison matters depends on the estimand: for the ATT, the question is whether the control scores cover the treated-score distribution; for the ATE, both arms must cover the full target population. Weight summaries — minimum, maximum, selected quantiles, and the coefficient of variation — expose instability directly. For matching, the bookkeeping of the design plays the same role: the number of unmatched treated units, the number of distinct controls used and how often each is reused, and the distribution of match distances.
The cost of extreme weights can be summarized by the effective sample size of a weighted arm with weights \(\{w_i\}\), \[n_{\mathrm{eff}} = \frac{\bigl(\sum_i w_i\bigr)^2}{\sum_i w_i^2},\] which equals the actual sample size when the weights are constant and collapses toward \(1\) when a single weight dominates. For two-arm estimators it is usually most informative arm by arm. A weighted arm with \(500\) units but \(n_{\mathrm{eff}} = 40\) delivers roughly the precision of \(40\) equally weighted observations; reporting \(n_{\mathrm{eff}}\) alongside the point estimate makes the price of near-overlap violations concrete.
6.3.6 Design Objectives Depend on the Estimand
The same propensity model need not serve every estimator equally, so the diagnostics above should be tailored to the target. For matching, the score must support close matches and covariate balance within the matched sample; the relevant diagnostics are match distances, residual SMDs among matched units, unmatched treated units, and control reuse. For ATE weighting, with weights \(1/\hat\pi(X_i)\) for treated and \(1/\{1-\hat\pi(X_i)\}\) for control units, both arms are re-weighted to the full target population, so weighted balance in both arms, weight tails, and arm-specific effective sample sizes are the primary concerns. For ATT weighting, with weights \[w_i = T_i + (1 - T_i)\,\frac{\hat\pi(X_i)}{1 - \hat\pi(X_i)},\] the treated arm is left unweighted and the central question is whether the controls can represent the treated units, so control coverage of the treated-score distribution is the key diagnostic.
The figure below displays the four weighting schemes as functions of the propensity score. The picture makes estimand targeting immediate: each scheme up-weights the units that must stand in for the population it targets, and instability appears exactly where a required weight diverges.
The general principle: a propensity model cannot be evaluated independently of the estimand and the estimator that will use it.
6.3.7 A Design-Stage Workflow
Propensity-score design is typically iterative: specify a treatment model, construct the proposed matched or weighted sample, assess balance and overlap, revise the design when the diagnostics are inadequate, and repeat. Revisions may include nonlinear terms, interactions, an alternative link or learner, regularization, a modified matching ratio or caliper, a different weighting target, or trimming. The outcome data remain hidden throughout this cycle.
Outcome blinding, in particular, does not mean that the specification is fixed after one fit. Three levels should be distinguished. Outcome-blind design: treatment-effect estimates are never inspected while the propensity model is being specified. Balance-driven revision: altering the model in response to observed imbalance is legitimate — indeed expected. Design lock: once balance and overlap are judged acceptable, the matching or weighting rule is fixed, and only then are outcomes analyzed. The benefit is not that subjectivity disappears; it is that design decisions cannot be selected because they produce a preferred causal estimate.
6.4 Matching on the Propensity Score
The propensity score also motivates matching: for each treated unit \(i\), find a control unit \(j\) with \(\hat\pi(X_j) \approx \hat\pi(X_i)\), and use \(Y_j\) as a local estimate of \(\E[Y(0) \mid \pi(X) = \pi(X_i)]\) (Abadie and Imbens 2006, 2016). It is important to be precise about what is being estimated: the matched control outcome \(Y_j\) is not a substitute for the individual counterfactual \(Y_i(0)\), which is unobservable and contains an irreducible idiosyncratic component. Rather, \(Y_j\) is a noisy local approximation to the conditional mean \(\E[Y(0) \mid \pi(X) = \pi(X_i)]\), which is identified from observed control outcomes via the chain \[\E[Y(0) \mid \pi(X) = \pi(X_i)] = \E[Y(0) \mid \pi(X) = \pi(X_i),\, T = 0] = \E[Y \mid \pi(X) = \pi(X_i),\, T = 0],\] where the first equality uses Theorem 6.2 and the second uses consistency. Averaging across treated units targets — and, under the regularity conditions discussed below, consistently estimates — the ATT: \[\hat\tau_{\mathrm{ATT}} = \frac{1}{n_1}\sum_{i:\, T_i=1} \bigl[Y_i - Y_{\hat\jmath(i)}\bigr], \tag{6.1}\] where \(n_1\) is the number of treated units and \(\hat\jmath(i)\) is the index of the matched control unit.
6.4.1 One-to-One Nearest Neighbor Matching
The most common matching algorithm is nearest-neighbor (1:1) matching (Abadie and Imbens 2006): for each treated unit \(i\), find \[\hat\jmath(i) = \arg\min_{j:\, T_j=0} \bigl|\hat\pi(X_i) - \hat\pi(X_j)\bigr|.\] When several controls are equally close, a convention is required — deterministic tie-breaking, averaging the outcomes of all equally close controls, or random selection; the choice affects the exact estimator and its variance, though not the identification logic. Matched control units may be selected with or without replacement. Matching with replacement typically reduces bias (each control unit is available to every treated unit, so match quality improves) but typically increases variance (a small number of control units may be used many times); neither direction is guaranteed without further conditions.
6.4.2 Caliper Matching
Caliper matching restricts matches to pairs within a maximum distance \(\delta\) (the “caliper”) on the propensity score scale. Treated units with no control match within the caliper are discarded. The caliper may be defined on the probability scale or — commonly in practice — on the logit scale \(\mathrm{logit}\,\hat\pi(X)\), which spaces units more evenly near the boundaries of \([0,1]\); the chosen scale must be reported, since numerical caliper values are not comparable across scales.
Because matchable treated units are retained and the rest discarded, the procedure no longer targets the original ATT: it targets an ATT restricted to the retained treated subpopulation with adequate overlap. And because the retained set depends on \(\hat\pi\), this target is data-dependent in finite samples; it stabilizes to a fixed population estimand only as \(\hat\pi\) converges.
6.5 Inverse Probability Weighting
6.5.1 The IPW Identification Formula
The ATE admits an alternative representation as an inverse probability weighting (IPW) formula, in which each observed outcome is weighted by the inverse probability of the treatment actually received.
Theorem 6.3 (IPW Identification) Fix \(t \in \{0, 1\}\) and assume: (i) consistency: \(T = t\) implies \(Y = Y(t)\); (ii) weak conditional exchangeability: \(Y(t) \indep T \mid X\); (iii) positivity: \(P(T{=}t \mid X) > 0\) almost surely; and (iv) integrability: \(\E|Y(t)| < \infty\). Then \[\E\!\left[\frac{T \cdot Y}{\pi(X)}\right] = \E[Y(1)], \qquad \E\!\left[\frac{(1-T) \cdot Y}{1 - \pi(X)}\right] = \E[Y(0)].\] If the conditions hold for both treatment levels, \[\tau_{\mathrm{ATE}} = \E\!\left[\frac{T \cdot Y}{\pi(X)} - \frac{(1-T) \cdot Y}{1 - \pi(X)}\right]. \tag{6.2}\]
Proof. \[\E\!\left[\frac{T Y}{\pi(X)}\right] = \E\!\left[\frac{T\, Y(1)}{\pi(X)}\right] = \E\!\left[\frac{Y(1)}{\pi(X)}\, \E\bigl[T \mid X,\, Y(1)\bigr]\right] = \E\!\left[\frac{Y(1)}{\pi(X)}\, \pi(X)\right] = \E[Y(1)].\] The first equality uses consistency (\(Y = Y(1)\) when \(T{=}1\)). The second is the law of iterated expectations conditioning on \((X, Y(1))\); the factor \(Y(1)/\pi(X)\) is measurable with respect to the conditioning variables and pulls out of the inner expectation. The third uses ignorability: \(Y(1) \indep T \mid X\) implies \(\E[T \mid X, Y(1)] = \pi(X)\). Integrability, with positivity, ensures each expectation is well defined. The case \(t = 0\) is symmetric. \(\square\)
6.5.2 The Horvitz–Thompson Estimator
The corresponding sample estimator is the Horvitz–Thompson (HT) IPW estimator: \[\hat\tau_{\mathrm{IPW}} = \frac{1}{n}\sum_{i=1}^n \left[\frac{T_i Y_i}{\hat\pi(X_i)} - \frac{(1-T_i) Y_i}{1-\hat\pi(X_i)}\right]. \tag{6.3}\] This estimator is consistent when \(\hat\pi(X)\) converges to \(\pi(X)\), given positivity and mild moment conditions — with flexible nuisance estimators evaluated on the same sample, additional conditions such as sample splitting may be required (Chapter 12) — but it can be inefficient due to extreme weights. In practice, a fitted score used for weighting is evaluated through weighted covariate balance, the weight distribution, and effective sample sizes.
The normalized (Hájek) estimator divides by the sum of weights within each group and is often more stable in practice: \[\hat\tau_{\mathrm{HJ}} = \frac{\sum_i T_i Y_i / \hat\pi(X_i)}{\sum_i T_i / \hat\pi(X_i)} - \frac{\sum_i (1-T_i) Y_i / (1-\hat\pi(X_i))}{\sum_i (1-T_i) / (1-\hat\pi(X_i))}. \tag{6.4}\] The efficient doubly robust estimator (AIPW) of Chapter 11 combines the outcome model and the propensity score; under correct nuisance specification it can achieve higher efficiency than either pure IPW or pure outcome regression alone.
6.5.3 Why the Weights Work: Weighted-Balance Identities
The balancing theorem and the IPW formula are two faces of the same mechanism, and the connection can be made algebraic: inverse-probability weights re-create, inside each treatment arm, the covariate distribution of the target population.
Lemma 6.2 (Weighted-Balance Identities) Let \(h(X)\) be any integrable function of the covariates and suppose \(0 < \pi(X) < 1\) almost surely. Then \[\E\!\left[\frac{T\, h(X)}{\pi(X)}\right] = \E[h(X)] \qquad\text{and}\qquad \E\!\left[\frac{(1-T)\, h(X)}{1-\pi(X)}\right] = \E[h(X)].\] Moreover, for the treated target population, \[\frac{1}{P(T{=}1)}\,\E\!\left[(1-T)\,\frac{\pi(X)}{1-\pi(X)}\, h(X)\right] = \E\bigl[h(X) \mid T{=}1\bigr].\]
Proof. For the first identity, condition on \(X\) and note that \(h(X)/\pi(X)\) pulls out of the inner expectation: \(\E[T h(X)/\pi(X)] = \E[h(X)\,\E(T \mid X)/\pi(X)] = \E[h(X)]\). The second is symmetric. For the third, \[\E\!\left[(1-T)\,\frac{\pi(X)}{1-\pi(X)}\, h(X)\right] = \E\!\left[\bigl\{1 - \pi(X)\bigr\}\,\frac{\pi(X)}{1-\pi(X)}\, h(X)\right] = \E\bigl[\pi(X)\, h(X)\bigr] = \E\bigl[T\, h(X)\bigr],\] and \(\E[T\, h(X)] = P(T{=}1)\,\E[h(X) \mid T{=}1]\). \(\square\)
Taking \(h \equiv 1\) shows that the average inverse weights in each arm have expectation one — the weight-sum quantities that reappear in the lab. Taking \(h(X) = X_j\) shows that the weighted treated and control samples reproduce the covariate means of the target population: the full population under the ATE weights, the treated subpopulation under the ATT odds weights applied to the control arm. This is why the weights take their particular forms. Note that no potential outcomes appear in the lemma: like the balancing theorem, it is a property of the assignment mechanism alone. Substituting the outcome \(Y\) for \(h(X)\) is what requires ignorability — that substitution is exactly the content of Theorem 6.3.
6.6 Overlap and Positivity
6.6.1 Positivity and Strong Overlap
We must distinguish two conditions on the propensity score that play different roles in identification and inference.
Together with unconfoundedness, positivity constitutes the strong ignorability of Rosenbaum and Rubin (1983). Positivity is the condition needed for identification: without it, certain strata have no counterfactual information and the ATE is not nonparametrically identified.
Strong overlap bounds the inverse weights uniformly away from infinity. It is a convenient sufficient condition for the standard \(\sqrt{n}\)-consistent, asymptotically normal theory of later chapters (Chapter 12), because the uniform bound provides a transparent regularity condition — but it is not necessary in every model: root-\(n\) inference can survive under weaker, estimator-specific tail and moment conditions, provided the relevant influence function retains finite variance. Positivity alone permits identification but does not guarantee stable inference: when \(\pi(X)\) approaches \(0\) or \(1\) in parts of the covariate space, IPW weights blow up and influence values become extreme, even though the parameter is technically identified (Khan and Tamer 2010).
Why positivity is necessary. The identification formula requires that both \(\E[Y \mid T{=}1, X{=}x]\) and \(\E[Y \mid T{=}0, X{=}x]\) be defined for \(P_X\)-almost all \(x\) in the support of \(X\). If \(\pi(x) = 0\) for some \(x\), then no treated units exist with \(X = x\), so \(\E[Y \mid T{=}1, X{=}x]\) is not estimable from the data. Similarly if \(\pi(x) = 1\).
6.6.2 Practical Consequences of Near-Violations
Exact violations of overlap are rare in practice. More common are near-violations: some covariate strata have very few treated or control units, making the estimated propensity score close to 0 or 1.
Near-violations arise when a covariate is a near-perfect predictor of treatment, when the covariate distributions of treated and control groups barely overlap, or when the propensity score model is over-flexible and fits the treatment variable nearly perfectly in-sample. Severe lack of overlap is therefore not necessarily a model-fitting failure: it may reflect a genuine absence of comparable treated and control units in the target population, in which case the honest responses are those of the next two subsections — trimming or re-targeting the estimand — rather than model revision.
6.6.3 Trimming Strategies
Trimming by propensity score. Restrict the analysis to units with estimated propensity scores in a prespecified interval \([\eta, 1-\eta]\) for some small \(\eta > 0\). The estimand changes to the ATE in the trimmed subpopulation: \[\tau_{\mathrm{trim}} = \E\bigl[Y(1) - Y(0) \mid \eta \le \pi(X) \le 1-\eta\bigr]. \tag{6.5}\] Note that the target population is defined by the true score \(\pi(X)\), while the practical rule is applied with the estimated score \(\hat\pi(X)\); the two coincide asymptotically when \(\hat\pi\) is consistent. The choice of \(\eta\) is a precision–target-population trade-off rather than a bias–variance trade-off in the usual sense: removing regions with extreme scores eliminates the largest weights, but it also discards observations and changes the estimand. The discrepancy between \(\tau_{\mathrm{trim}}\) and the original ATE is a change of target, not a statistical bias relative to \(\tau_{\mathrm{trim}}\), and the net variance need not decrease monotonically in \(\eta\).
Crump et al. (2009) rule. Crump et al. (2009) derive an optimal symmetric trimming threshold — distribution-dependent, minimizing the asymptotic variance of the trimmed ATE estimator under their criterion. Cutoffs near \(0.1\) often emerge from this calculation, so the fixed rule \(\eta = 0.1\) is a common heuristic, though not a universal optimum.
6.6.4 Re-targeting the Estimand to Avoid Overlap Failures
Trimming restricts the population in order to preserve the ATE within a region of good overlap. An alternative, complementary response is to leave the population alone but change the estimand: target the average treatment effect on a subset of the population for which the relevant side of positivity can be guaranteed.
ATT (target the treated population). The ATT averages \(Y(1) - Y(0)\) over \(X \mid T{=}1\) and therefore requires only that \(\E[Y(0) \mid X = x]\) be identified where the treated population lives. The treated-covariate law is absolutely continuous with respect to \(P_X\) with density proportional to the propensity score, \(dP_{X \mid T=1} = \{\pi(x)/P(T{=}1)\}\, dP_X\), so treated-population mass concentrates on \(\{x : \pi(x) > 0\}\); there the requirement reduces to \(\pi(x) < 1\). Hence, for nonparametric identification by adjustment: \[\tau_{\mathrm{ATT}} \text{ is identified iff } \pi(X) < 1 \text{ a.s. on the support of } X\mid T{=}1.\] The ATT therefore tolerates regions with \(\pi(x) = 0\) (no treated units are observed there in any case); it cannot tolerate regions inside the treated support where \(\pi(x) = 1\) (no comparable controls).
ATC (target the control population). Symmetrically, the ATC averages over \(X \mid T{=}0\) and is nonparametrically identified by adjustment iff \(\pi(X) > 0\) a.s. on the support of \(X \mid T{=}0\). These equivalences are statements about nonparametric identification by adjustment, under consistency and the relevant arm-specific ignorability condition; additional outcome-model restrictions could identify the same estimands under weaker support conditions.
Practical rule. When overlap fails at covariate values where \(\pi(X) \approx 0\) — treated units are scarce or absent there — re-target to the ATT. When overlap fails where \(\pi(X) \approx 1\) — controls are scarce or absent — re-target to the ATC. When overlap fails on both sides, restrict to the trimmed-population estimand.
6.7 Lab: Simulation Study of IPW and Matching Estimators
This lab compares five estimators on a five-covariate data-generating process with treatment-effect heterogeneity. With five continuous confounders, direct stratification is infeasible, making propensity-score methods the natural choice. The estimators span two estimands: the ATE (targeted by the HT and Hájek IPW estimators) and the ATT (targeted by the ATT-IPW variants and by matching). Five questions drive the comparison:
- How much variance does weight normalization save (Hájek over HT) for ATE estimation, and why does the baseline outcome mean matter?
- Does the same variance problem afflict HT when targeting the ATT, and does Hájek normalization help as much?
- Among estimators targeting the ATT, do IPW and matching agree on the target, and which is more efficient?
- How large is the estimand gap between ATT and ATE, and which estimators reveal it?
- How does replacing the true propensity score with a logistic estimate change bias and variance?
6.7.1 Part 1: Correctly Specified Propensity Score
With five confounders, direct cell-by-cell stratification is infeasible. The propensity score reduces this five-dimensional adjustment to a scalar. The large baseline mean (\(\E[Y(0)] = 8\)) plays a distinct role: it amplifies the consequences of extreme IPW weights, making Hájek normalization strongly recommended in this setting rather than merely helpful.
Estimators. Five estimators are compared, each applied under both the true and estimated propensity score. The estimated score is obtained by logistic regression of \(T\) on \((X_1, \ldots, X_5)\), which is correctly specified since the true logit is linear in all five covariates.
Horvitz–Thompson ATE (HT-ATE), Equation 6.3. Targets the ATE. Raw inverse weights can be extreme when \(\hat\pi(X_i)\) is near \(0\) or \(1\), and with \(\E[Y(0)]=8\) the variance consequences are severe.
Hájek ATE (HJ-ATE), Equation 6.4. Targets the ATE. Normalizing by the empirical weight sum within each arm caps the effective influence of any single unit.
Horvitz–Thompson ATT (HT-ATT). The ATT identification formula re-weights the control arm to look like the treated population while leaving the treated arm unchanged: \[\hat\tau_{\mathrm{HT,ATT}} = \frac{1}{n_1}\sum_{i:\,T_i=1} Y_i \;-\; \frac{1}{n_1}\sum_{i:\,T_i=0} \frac{\hat\pi(X_i)}{1-\hat\pi(X_i)}\,Y_i.\] Each control unit receives weight proportional to its odds of treatment: units whose covariates resemble the treated population receive large weight. For a control unit with \(\hat\pi(X_i) = 0.95\) the weight is \(19\); combined with a baseline outcome near \(8\), this produces extreme variance.
Hájek ATT (HJ-ATT). The normalized counterpart replaces the raw-weight sum over the control arm with a self-normalized version: \[\hat\tau_{\mathrm{HJ,ATT}} = \frac{1}{n_1}\sum_{i:\,T_i=1} Y_i \;-\; \frac{\sum_{i:\,T_i=0} \{\hat\pi(X_i)/(1-\hat\pi(X_i))\}\,Y_i}{\sum_{i:\,T_i=0} \hat\pi(X_i)/(1-\hat\pi(X_i))}.\] The treated arm is already a simple mean and requires no normalization. This is the recommended default estimator for the ATT in practice.
One-to-one nearest-neighbor matching with replacement (NNM), Equation 6.1. Targets the ATT. Matching with replacement ensures every treated unit finds its globally closest control, preventing the finite-sample “running out of good matches” problem that arises without replacement when the propensity score distribution is wide (Abadie and Imbens 2006).
Results (\(n = 1{,}000\), \(B = 2{,}000\) replications, seed 42). Bias is computed relative to each estimator’s own target: ATE \(= 1.000\) for HT-ATE and HJ-ATE; ATT \(\approx 1.199\) for HT-ATT, HJ-ATT, and NNM. Monte Carlo standard errors of the reported means equal \(\mathrm{SD}/\sqrt{B} \approx \mathrm{SD}/45\).
| PS | Estimator | Estimand | Mean | Bias | SD | RMSE |
|---|---|---|---|---|---|---|
| Known | HT-ATE | ATE | 1.008 | +0.008 | 0.626 | 0.626 |
| Known | HJ-ATE | ATE | 1.003 | +0.003 | 0.145 | 0.145 |
| Known | HT-ATT | ATT | 1.184 | −0.016 | 0.725 | 0.725 |
| Known | HJ-ATT | ATT | 1.203 | +0.003 | 0.131 | 0.131 |
| Known | NNM | ATT | 1.207 | +0.008 | 0.130 | 0.130 |
| Estimated | HT-ATE | ATE | 0.998 | −0.002 | 0.194 | 0.194 |
| Estimated | HJ-ATE | ATE | 1.002 | +0.002 | 0.102 | 0.103 |
| Estimated | HT-ATT | ATT | 1.202 | +0.002 | 0.251 | 0.251 |
| Estimated | HJ-ATT | ATT | 1.202 | +0.002 | 0.101 | 0.101 |
| Estimated | NNM | ATT | 1.205 | +0.006 | 0.121 | 0.121 |
1. Hájek normalization is strongly recommended for ATE estimation when outcomes are non-centered. With the known propensity score, HT-ATE achieves SD \(= 0.626\), while HJ-ATE achieves SD \(= 0.145\) — a 4.3-fold reduction from simply normalizing the weights. The mechanism is transparent: a unit with \(\pi(X_i) = 0.05\) receives raw weight \(20\); multiplied by an outcome near \(\E[Y(0)] = 8\), its contribution to the HT sum is of order \(160\). Variability in which units are extreme across samples inflates the HT standard deviation to more than half the true effect size.
The role of the baseline can be isolated exactly. Define the average inverse weights in each arm, \[w_T = \frac{1}{n}\sum_{i=1}^n \frac{T_i}{\hat\pi(X_i)}, \qquad w_C = \frac{1}{n}\sum_{i=1}^n \frac{1 - T_i}{1 - \hat\pi(X_i)},\] each of which has expectation one under the true score (Lemma 6.2 with \(h \equiv 1\)) but fluctuates in finite samples. Shifting every outcome by a constant \(c\) changes the HT estimator by \[\hat\tau_{\mathrm{HT}}(Y + c) = \hat\tau_{\mathrm{HT}}(Y) + c\,(w_T - w_C),\] whereas the Hájek difference is exactly invariant to the shift, since each arm’s self-normalized mean moves by \(c\) and the shifts cancel. With a baseline near \(c = 8\), finite-sample fluctuation in \(w_T - w_C\) is amplified eightfold — this, not variability in the treatment effect itself, is the dominant source of the HT variance here.
2. HT-ATT is even more unstable; Hájek normalization is equally beneficial for ATT estimation. With the known PS, HT-ATT achieves SD \(= 0.725\), slightly worse than HT-ATE. The reason is structural: the ATT re-weights the control arm by the treatment odds \(\pi(X)/(1-\pi(X))\). A control unit with \(\pi(X) = 0.95\) receives an odds-ratio weight of \(19\); combined with the large baseline outcome this is as explosive as the ATE inverse-probability weight. The apparent bias of \(-0.016\) for HT-ATT is within one Monte Carlo standard error of \(0.725/\sqrt{2000} \approx 0.016\) and should be attributed to simulation noise rather than systematic inconsistency. HJ-ATT reduces SD to \(0.131\) — a 5.5-fold reduction — confirming that Hájek normalization is not an ATE-specific fix but a general principle applicable whenever inverse-probability-style weights are used.
3. HJ-ATT and NNM converge to the same target with nearly identical efficiency. Under the known PS, both target the ATT \(\approx 1.199\) and achieve essentially the same standard deviation (0.131 vs. 0.130). This is the sharpest demonstration of the “two routes, one estimand” principle: IPW re-weights the full sample using the propensity score; matching discards some control units and retains only those nearest each treated unit on the propensity scale. Both are consistent for the ATT under strong ignorability, and at \(n = 1{,}000\) they are virtually indistinguishable in efficiency. The practical implication is that the choice between IPW-ATT and matching can be made on secondary grounds — interpretability, balance diagnostics, or computational cost — rather than efficiency when the PS is known.
4. The estimand gap between ATE and ATT is large and clearly revealed by the table. The ATE estimators converge to \(1.000\); the ATT estimators converge to \(\approx 1.199\). The \(0.199\) gap is not a bias — it is a real difference in the estimands. High-\(X_1\) units are more likely to be treated (Equation 6.6) and have larger treatment effects (Equation 6.7); the treated subpopulation therefore benefits more than the full population average. A researcher who applies NNM and reports its output against the ATE benchmark of \(1.000\) would conclude the estimator has \(20\%\) bias; in fact it is unbiased for the correct target. Declaring the estimand before looking at any results is a non-negotiable discipline.
5. Estimated PS collapses HT variance and makes HJ-ATT the most efficient ATT estimator. Under the estimated PS, HT-ATE SD falls from \(0.626\) to \(0.194\) (69% reduction); HT-ATT SD falls from \(0.725\) to \(0.251\) (65% reduction). This counterintuitive gain — the estimated score outperforming the true one — is a known phenomenon, but it is not a consequence of consistency alone: the classical result requires a correctly specified propensity model estimated by maximum likelihood (here, the logistic model is exactly correct). The likelihood score equations of the logistic fit, \(\sum_i (T_i - \hat\pi(X_i))X_i = 0\), act as control variates: they remove the component of the IPW sampling error correlated with the score functions. IPW with the ML estimate of a correctly specified score is therefore asymptotically no less efficient than IPW with the known score, and an analogous conclusion holds for suitable nonparametric score estimators (Hirano et al. 2003).
More strikingly, HJ-ATT with estimated PS achieves SD \(= 0.101\) — beating both NNM (SD \(= 0.121\)) and HT-ATT (SD \(= 0.251\)) by a wide margin and matching HJ-ATE (SD \(= 0.102\)) almost exactly. This is the finite-sample analog of the efficiency result of Hirano et al. (2003): a properly normalized IPW estimator for the ATT is consistent — though, as a ratio estimator, not exactly unbiased in finite samples — and can be more efficient than matching when the propensity score model is well estimated.
6.7.2 Part 2: Robustness to PS Model Misspecification
Part 1 established that under a correctly specified PS model, IPW and NNM are both consistent for their respective estimands, with HJ-ATT and NNM delivering similar efficiency. Part 2 breaks that assumption by introducing a nonlinear true propensity score that the standard logistic regression cannot capture.
Modified DGP. The outcome model is unchanged from Equation 6.7. The true propensity score now contains a quadratic term in \(X_1\), and \(X_2\) and \(X_3\) do not affect treatment: \[\pi^*(X) = \expit\!\bigl(-0.5 + 0.8X_1 + 0.5X_1^2 + 0.2X_4 + 0.1X_5\bigr). \tag{6.8}\] The quadratic \(0.5X_1^2\) means that both large positive and large negative values of \(X_1\) increase the probability of treatment, creating a U-shaped propensity surface. Because \(\E[X_1] = 0\) and the CATE is still \(\tau(X) = 1 + 0.5X_1\), the ATE is unchanged at \(1.000\) exactly. The ATT changes because the new PS shifts which units are treated: \(\tau_{\mathrm{ATT}}^* \approx 1.152\) (oracle run, \(n = 10^7\)).
The estimated PS is still obtained by fitting a linear logistic regression of \(T\) on \((X_1, \ldots, X_5)\). This model is misspecified: it omits the \(X_1^2\) term and includes \(X_2\) and \(X_3\), which do not appear in the true logit.
| PS | Estimator | Estimand | Mean | Bias | SD | RMSE |
|---|---|---|---|---|---|---|
| True | HT-ATE | ATE | 1.028 | +0.028 | 0.824 | 0.825 |
| True | HJ-ATE | ATE | 1.013 | +0.013 | 0.152 | 0.152 |
| True | HT-ATT | ATT | 1.186 | +0.034 | 1.333 | 1.333 |
| True | HJ-ATT | ATT | 1.183 | +0.031 | 0.212 | 0.215 |
| True | NNM | ATT | 1.178 | +0.026 | 0.175 | 0.177 |
| Misspecified | HT-ATE | ATE | 1.478 | +0.478 | 0.147 | 0.501 |
| Misspecified | HJ-ATE | ATE | 0.861 | −0.139 | 0.087 | 0.164 |
| Misspecified | HT-ATT | ATT | 1.780 | +0.628 | 0.162 | 0.649 |
| Misspecified | HJ-ATT | ATT | 1.306 | +0.154 | 0.081 | 0.174 |
| Misspecified | NNM | ATT | 1.172 | +0.020 | 0.138 | 0.139 |
6. PS misspecification removes the consistency guarantee for IPW; here every IPW estimator is materially biased, and the Hájek correction cannot fix this. Under the misspecified PS, HT-ATE bias is \(+0.478\), nearly half the true effect; HJ-ATE bias is \(-0.139\). For the ATT estimators the problem is worse: HT-ATT bias is \(+0.628\); HJ-ATT bias is \(+0.154\). Comparing these to the near-zero biases in the “True PS” rows makes clear that the Hájek normalization is a variance correction — it cannot compensate for a misspecified model that produces systematically wrong weights. IPW consistency is tied to consistency of the PS estimator; a structurally wrong model generally produces structurally wrong weights, and no normalization of wrong weights produces correct estimates. (Special calibration identities can leave particular weighted estimators consistent under particular misspecifications, but no such protection is generic, and none operates here.)
Notice also that HJ-ATE and HT-ATE have biases in opposite directions (\(-0.139\) vs. \(+0.478\)). The mechanism is a multiplicative level error that HT inherits from misspecification but Hájek removes. Recall the average inverse weights \(w_T\) and \(w_C\), each of which converges to \(1\) under a correctly specified PS. Under misspecification both can drift; in this DGP the Monte Carlo means are \(w_T \approx 1.045\) and \(w_C \approx 0.972\). Algebraically, \(\hat\tau_{\mathrm{HT}} = w_T\,\hat\mu_1^{\mathrm{HJ}} - w_C\,\hat\mu_0^{\mathrm{HJ}}\), so the HT bias inherits an extra contribution \[(w_T - 1)\,\E[Y(1)] \;-\; (w_C - 1)\,\E[Y(0)] \;\approx\; 0.045 \times 9 \;+\; 0.028 \times 8 \;\approx\; +0.63,\] which accounts for essentially the entire gap between the two biases (\(+0.478 - (-0.139) = +0.617\)). With \(\E[Y(0)] = 8\), even sub-five-percent weight-sum errors translate into bias contributions of order \(0.3\). What remains in HJ-ATE is a smaller shape error: the residual misweighting of the population that survives self-normalization. The level and shape errors are determined by different aspects of the misspecification — the first by how far the average inverse weights drift from one, the second by how the misspecified weights re-weight the population — so there is no reason for their signs to align.
7. In this simulation, matching is less sensitive to the particular PS misspecification. The NNM bias barely changes: \(+0.026\) under the true PS versus \(+0.020\) under the misspecified PS. Its RMSE actually improves slightly (\(0.177 \to 0.139\)) because the misspecified logistic scores, being shrunk toward the mean of \(T\), produce somewhat less variable matches than the true nonlinear scores do. The reason is that NNM does not require the PS model to be correct in order to produce good matches — it only requires that the estimated score roughly orders units so that matched pairs are approximately balanced on the true confounders. The linear logistic model, though wrong, still captures the dominant linear effects of \(X_1\), \(X_4\), and \(X_5\) on treatment; the quadratic component is missed, but the ranking of units is not drastically distorted.
This insensitivity is conditional on the estimated score preserving covariate balance. Propensity-score matching can also fail badly under misspecification — if, for example, the misspecification distorts the ranking of units enough that matched pairs are imbalanced on outcome-relevant covariates. The lesson is therefore not that matching is unconditionally robust, but that its bias depends on covariate balance after matching rather than on the parametric correctness of the score itself; balance must be checked, not assumed. Theoretical analyses of when this conditional robustness holds appear in Yang et al. (2016) and Yang and Zhang (2023).
8. In this simulation, the relative advantage of matching over IPW reverses with model misspecification. In Part 1, HJ-ATT was more efficient than NNM under the estimated PS (SD \(= 0.101\) vs. \(0.121\)). In Part 2, NNM dominates on both bias and RMSE relative to HJ-ATT. The practical implication is that the efficiency advantage of IPW is contingent on the PS model being approximately correct. When there is reason to doubt the functional form of the propensity model — for instance, when treatment is driven by a continuous covariate in a nonlinear way — matching can be a more robust alternative, because it relies on local covariate similarity rather than a globally correct weighting function; whether this conditional advantage materializes depends on whether the estimated score still produces close matches on the outcome-relevant covariates.
| Part 1: correct PS model, bias | RMSE | Part 2: misspecified PS model, bias | RMSE | |
|---|---|---|---|---|
| HJ-ATT | \(+0.002\) | \(0.101\) | \(+0.154\) | \(0.174\) |
| NNM | \(+0.006\) | \(0.121\) | \(+0.020\) | \(0.139\) |
6.8 The Limits of Propensity Score Methods
6.8.1 The Untestable Assumption
The balancing property (Theorem 6.1), the coarsest-balancing-score lemma, and the weighted-balance identities (Lemma 6.2) are properties of the observed treatment-assignment mechanism: they hold for any binary treatment and require no causal assumptions. The causal results of this chapter — the Propensity Score Theorem, the IPW identification formula, the causal interpretation of matching — additionally rest on the unconfoundedness assumption \((Y(0), Y(1)) \indep T \mid X\), together with consistency and the relevant positivity condition.
Unconfoundedness requires that the observed pretreatment set \(X\) be sufficient for adjustment. In graphical terms: \(X\) blocks every back-door path from \(T\) to \(Y\). This does not require that every latent common cause itself be observed: an observed variable lying on a confounding path can block that path even when the common cause behind it is latent (Chapter 3). What is required is that every noncausal path be intercepted by the observed set — and this can never be verified from the data alone.
Two complementary responses. When unconfoundedness is in doubt, two distinct strategies are available and they answer different questions. Sensitivity analysis (Chapter 9) keeps the back-door framework but quantifies how strong an unmeasured confounder would have to be to overturn the conclusion: if the answer is “implausibly strong” the analysis is robust, whereas if the answer is “modest” the conclusion is fragile. The instrumental variable approach (Chapter 7) replaces the unconfoundedness assumption with a different set of identifying assumptions, exploiting an external source of variation in treatment that is independent of the unobserved treatment–outcome confounders. The hidden-confounding concern is thereby relocated, not eliminated: the instrument itself must be valid — unconfounded with the outcome and excluded from affecting it directly.
6.8.2 The Road to Instrumental Variables
The limitation of back-door adjustment motivates the search for alternative identification strategies when unobserved confounders are present. One canonical alternative — the one taken up next — is the instrumental variable (IV), studied in Chapter 7. It is not the only one: front-door identification (Chapter 3) and sensitivity analysis with partial identification (Chapter 9) are others.
| Strategy | Key assumption | Identification mechanism |
|---|---|---|
| Back-door adjustment (propensity score) | Observed pretreatment set blocks all back-door paths | Condition on \(X\) to block \(T \leftarrow U \rightarrow Y\) |
| Instrumental variables | Some confounders may be unobserved | Exploit exogenous variation \(Z \rightarrow T\) |
Back-door adjustment removes confounding by conditioning on observed variables. Instrumental variables take a different approach: instead of blocking the confounding path \(T \leftarrow U \rightarrow Y\), they isolate variation in \(T\) induced by an external source \(Z\) that is independent of \(U\). Under additional IV assumptions, the ratio of the \(Z\)-induced change in \(Y\) to the \(Z\)-induced change in \(T\) — the Wald estimator — identifies a causal parameter. In the binary instrument and treatment setting, under the additional monotonicity (no-defiers) assumption, that parameter is the local average treatment effect (LATE) among compliers. This is derived formally in Chapter 7.
6.9 Summary
The propensity score reduces dimension. The propensity score \(\pi(X) = P(T{=}1 \mid X)\) is a balancing score — \(T \indep X \mid \pi(X)\) — for any treatment-assignment mechanism, with no causal assumption required. Under unconfoundedness and positivity, the balancing property upgrades to \((Y(0),Y(1)) \indep T \mid \pi(X)\) (Theorem 6.2), so identification requires adjustment for the scalar \(\pi(X)\) alone, not the full vector \(X\).
IPW identification. Under consistency, weak conditional exchangeability for both treatment levels, positivity, and integrability, the ATE is identified by Equation 6.2 (Theorem 6.3); joint strong ignorability supplies the exchangeability and positivity components but is stronger than necessary, and consistency remains a separate assumption. The Horvitz–Thompson IPW estimator is consistent but can be sensitive to extreme weights; the Hájek variant is often more stable in finite samples. Augmented IPW (AIPW) can achieve higher efficiency under correct nuisance specification, as established formally in Chapter 11.
The matching procedure in this chapter targets the ATT. The nearest-neighbor matching estimator presented here finds control units with similar propensity scores to each treated unit. It is designed to target the ATT and provides transparent covariate balance diagnostics, but differs fundamentally from IPW weighting in estimand and methodology. More generally, matching procedures can be designed to target the ATT, ATC, or an ATE-like estimand depending on the matching design and weighting scheme.
Positivity is necessary; strong overlap is a standard sufficient condition for stable estimation. Positivity is necessary for identification of the ATE. Strong overlap is a stronger, convenient sufficient condition under which the standard \(\sqrt{n}\)-consistent, asymptotically normal theory goes through. Near-violations of strong overlap create extreme IPW weights and large variance; trimming strategies address this at the cost of changing the estimand.
The fundamental limitation. Unconfoundedness is a strong, untestable assumption. When unobserved confounders are present, propensity score adjustment fails. An instrumental variable is one important alternative identification strategy (Chapter 7); others include front-door identification (Chapter 3) and sensitivity analysis with partial identification (Chapter 9).
6.10 Problems
1. Propensity score and balancing. Suppose \(X = (X_1, X_2)\), where \(X_1\) is binary and \(X_2\) is continuous. The propensity score model is correctly specified as \(\mathrm{logit}(\pi(X)) = \beta_0 + \beta_1 X_1 + \beta_2 X_2\).
- State and prove the Balancing Property \(T \indep X \mid \pi(X)\).
- Two units \(i\) and \(j\) have \(X_i = (1, 2.3)\) and \(X_j = (0, 3.5)\) but \(\pi(X_i) = \pi(X_j) = 0.4\) under the true propensity score. Explain what equality of the true propensity score implies about the conditional distribution of \(X\) across treatment arms at this score value, and why this is a population-level balancing statement rather than an exact individual counterfactual equality: if \(i\) is treated and \(j\) is control, \(Y_j\) is a noisy estimate of a conditional mean, not of \(Y_i(0)\). State the additional assumptions needed for propensity-score matching to consistently estimate the ATT.
- Explain why matching on \(\hat\pi(X)\) is not the same as matching on \(X\) directly. Under what conditions do they give the same answer?
2. ATE, ATT, and propensity score weighting. A dataset has \(n = 1000\) observations with estimated propensity scores \(\hat\pi(X_i)\) and \(n_1 = 400\) treated units.
- Write the Horvitz–Thompson IPW estimator of the ATE as a weighted sum of observed outcomes. What weights do treated units receive? What weights do control units receive?
- Derive an analogous IPW estimator for the ATT. (Hint: the ATT averages over the treated distribution; the denominator of the weight for control units should reflect this.)
- Suppose the propensity score is constant: \(\pi(X) = p\) for all units. Show that the Horvitz–Thompson IPW estimator of the ATE equals \[\frac{n_1}{np}\,\bar Y_1 - \frac{n_0}{n(1-p)}\,\bar Y_0,\] and conclude that it reduces to the difference in means only when the realized count satisfies \(n_1 = np\) exactly — as under complete randomization with fixed arm sizes — whereas under Bernoulli assignment \(n_1\) is random and the equality fails. Show that the Hájek estimator reduces to \(\bar Y_1 - \bar Y_0\) exactly whenever the score is constant. Under which experimental designs does \(\pi(X) = 0.5\) hold for each unit: Bernoulli randomization with \(P(T_i{=}1) = 0.5\), complete randomization with exactly \(n/2\) treated, or both?
3. Overlap. Let \(\pi(X)\) be the true propensity score and suppose overlap fails on a set \(\mathcal{S}_1\) of covariate values with \(P(X \in \mathcal{S}_1) > 0\): \(\pi(x) = 1\) for all \(x \in \mathcal{S}_1\). (For a continuous covariate, a violation at a single \(P_X\)-null point would be harmless; positivity is a statement about non-null sets.)
- For each of \(\tau_{\mathrm{ATE}}\) and \(\tau_{\mathrm{ATT}}\), identify the exact formula that fails to be identified, and explain why. (Hint: consider whether \(\mathcal{S}_1\) intersects the support of \(X \mid T{=}1\).)
- Suppose overlap fails only on a set \(\mathcal{S}\) with \(P(X \in \mathcal{S}) = 0.15\). Define the trimmed ATE estimand. How does it differ from the ATE?
- A researcher applies the Crump et al. (2009) rule with \(\eta = 0.1\), removing 8% of the sample. (i) List two reasons the trimmed estimator may have lower variance than the untrimmed IPW estimator. (ii) What is the cost in terms of external validity?
4. Covariate selection for the propensity model. A researcher has pretreatment covariates \(C\), \(P\), and \(Z\), and a post-treatment variable \(M\). In the assumed causal graph, \(C\) is a confounder of \(T\) and \(Y\); \(P\) predicts the outcome but not treatment; \(Z\) strongly predicts treatment but has no causal relation to the potential outcomes; and \(M\) is a mediator on the path from \(T\) to \(Y\).
- Which variables belong in the propensity model when the target is the total effect of \(T\) on \(Y\)? Justify each inclusion or exclusion using the taxonomy of Section 6.3.1.
- Explain how including \(Z\) may affect overlap, the distribution of the inverse-probability weights, and the effective sample size, even though it does not invalidate the adjustment set.
- Suppose the researcher discovers that \(C\) was in fact measured after treatment for a subset of units. What concerns does this raise for the propensity design?
5. Hidden confounding (conceptual preview of Chapter 9). Suppose you estimate \(\hat\tau_{\mathrm{ATE}} = 3.2\) using propensity-score methods, and a referee questions whether unconfoundedness holds.
- Define what it means for \(U\) to be a “hidden confounder” in the context of the DAG, and explain why its presence invalidates the IPW identification formula.
- Explain in words what it means for an estimated effect to be “robust to hidden confounding,” and why such an assessment is necessarily comparative: it depends on a quantitative yardstick for how strong an unmeasured confounder would have to be to overturn the conclusion. Without doing formal calculations, sketch the narrative a researcher might construct to argue that \(\hat\tau_{\mathrm{ATE}} = 3.2\) would survive any plausible hidden confounder. (Chapter 9 develops three formal yardsticks: Rosenbaum’s \(\Gamma\), the E-value, and the marginal sensitivity model.)
- How can a valid instrumental variable identify a causal parameter despite unmeasured confounding between \(T\) and \(Y\) (Chapter 7)? State the assumptions that replace treatment–outcome unconfoundedness, and explain why unmeasured confounding between the instrument and the outcome would still invalidate the IV design.
6. Matching vs. IPW. You have \(n = 500\) observations with a binary treatment and a scalar propensity score. You compare (A) the Hájek IPW estimator targeting the ATE and (B) 1:1 nearest-neighbor matching designed to target the ATT.
- Explain in one sentence why estimator (A) and estimator (B) target different estimands even though both use \(\hat\pi(X)\).
- In your matched sample, the standardized mean difference for covariate \(X_1\) is 0.05 after matching and 0.45 before matching. What does this tell you about the success of matching, and what assumption does balance on observed covariates not verify?
- Under what condition on the treatment effect heterogeneity are the ATE and ATT equal?
Abadie, Alberto, and Guido W. Imbens. 2006. “Large Sample Properties of Matching Estimators for Average Treatment Effects.” Econometrica 74 (1): 235–67.
Abadie, Alberto, and Guido W. Imbens. 2008. “On the Failure of the Bootstrap for Matching Estimators.” Econometrica 76 (6): 1537–57. https://doi.org/10.3982/ECTA6474.
Abadie, Alberto, and Guido W. Imbens. 2016. “Matching on the Estimated Propensity Score.” Econometrica 84 (2): 781–807. https://doi.org/10.3982/ECTA11293.
Brookhart, M. Alan, Sebastian Schneeweiss, Kenneth J. Rothman, Robert J. Glynn, Jerry Avorn, and Til Stürmer. 2006. “Variable Selection for Propensity Score Models.” American Journal of Epidemiology 163 (12): 1149–56. https://doi.org/10.1093/aje/kwj149.
Crump, Richard K., V. Joseph Hotz, Guido W. Imbens, and Oscar A. Mitnik. 2009. “Dealing with Limited Overlap in Estimation of Average Treatment Effects.” Biometrika 96 (1): 187–99. https://doi.org/10.1093/biomet/asn055.
Hirano, Keisuke, Guido W. Imbens, and Geert Ridder. 2003. “Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score.” Econometrica 71 (4): 1161–89. https://doi.org/10.1111/1468-0262.00442.
Imbens, Guido W., and Donald B. Rubin. 2015. Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press.
Khan, Shakeeb, and Elie Tamer. 2010. “Irregular Identification, Support Conditions, and Inverse Weight Estimation.” Econometrica 78 (6): 2021–42.
Rosenbaum, Paul R., and Donald B. Rubin. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika 70 (1): 41–55.
VanderWeele, Tyler J. 2019. “Principles of Confounder Selection.” European Journal of Epidemiology 34 (3): 211–19. https://doi.org/10.1007/s10654-019-00494-6.
Yang, Shu, Guido W. Imbens, Zhanglin Cui, Douglas E. Faries, and Zbigniew Kadziola. 2016. “Propensity Score Matching and Stratification in Observational Studies with Multi-Level Treatments.” Biometrics 72: 1055–65.
Yang, Shu, and Yunshu Zhang. 2023. “Multiply Robust Matching Estimators of Average and Quantile Treatment Effects.” Scandinavian Journal of Statistics 50: 235–65.