6 Propensity Score Methods
6.1 Motivation: The Curse of Dimensionality
Chapter 5 established that under consistency, conditional exchangeability, positivity, and integrability, the ATE is identified by the back-door adjustment formula, and developed three estimation strategies: regression adjustment, standardization, and stratification. All three require, in practice, that we condition on a covariate vector \(X\). When \(X\) is low-dimensional, this is feasible. When \(X\) has many components, it is not.
The problem. Stratification requires cells \(\{X = x\}\) to contain both treated and control units. With \(p\) binary covariates, there are \(2^p\) cells. With \(p = 10\), that is 1024 cells — far more than most datasets can support. Regression adjustment avoids the cell-count problem but requires a correctly specified model for \(\E[Y \mid T, X]\), which becomes harder to specify reliably as \(p\) grows.
The solution. Rosenbaum and Rubin (1983) showed that it is sufficient to condition on a single scalar function of \(X\): the propensity score \(\pi(X) = P(T{=}1 \mid X)\). The propensity score reduces the adjustment dimension without changing the identified estimand, provided unconfoundedness and positivity hold. Identification power is preserved; finite-sample efficiency, covariate balance, and robustness to model misspecification, however, depend on how well the score is estimated and how it is used in the downstream estimator.
Example 6.1 (A Running Example: Four Cells, Three Propensity Values) Throughout the chapter we return to an idealized population of \(N = 1{,}000\) units with two binary covariates \(X = (X_1, X_2)\). Read the proportions in the table as probabilities and \(\mu(t,x) = \E\{Y(t) \mid X = x\}\) as population conditional means; under ignorability, \(\mu(t,x) = \E(Y \mid T{=}t, X{=}x)\).
| \(x=(x_1,x_2)\) | \(N_x\) | \(N_{1x}\) | \(N_{0x}\) | \(\pi(x)\) | \(\mu(1,x)\) | \(\mu(0,x)\) | \(\tau(x)\) |
|---|---|---|---|---|---|---|---|
| \((0,0)\) | 400 | 80 | 320 | 0.2 | 5 | 3 | 2 |
| \((1,0)\) | 200 | 100 | 100 | 0.5 | 9 | 6 | 3 |
| \((0,1)\) | 200 | 100 | 100 | 0.5 | 7 | 6 | 1 |
| \((1,1)\) | 200 | 160 | 40 | 0.8 | 12 | 8 | 4 |
The targets are \(\tau_{\mathrm{ATE}} = \sum_x P(x)\tau(x) = 2.4\) and \(\tau_{\mathrm{ATT}} = \sum_x P(x \mid T{=}1)\tau(x) = 30/11 \approx 2.73\), while the naive comparison of arm means is \(\bar Y_1 - \bar Y_0 = 8.91 - 4.43 = 4.48\): treated units sit in cells with large \(\mu\). Four cells, but only three values of \(\pi\): the cells \((1,0)\) and \((0,1)\) share \(\pi = 0.5\) although their conditional means differ. Whether, and in what sense, these two cells may be pooled is the question the next section answers.
6.2 The Propensity Score and Its Balancing Properties
The central insight of Rosenbaum and Rubin (1983) is that the \(p\)-dimensional adjustment problem of Chapter 5 reduces to a one-dimensional one. The right abstraction is the balancing score: any function of \(X\) that renders treatment assignment conditionally independent of the covariates. The propensity score is the canonical — and coarsest — such function.
6.2.1 Balancing Scores and the Propensity Score
Definition 6.1 (Balancing Score (Rosenbaum and Rubin 1983)) A function \(b(X)\) is called a balancing score if \[T \indep X \mid b(X).\]
The full covariate vector \(X\) is trivially a balancing score: \(T \indep X \mid X\) holds because, conditional on \(X\), the conditional distribution of \(X\) is degenerate and therefore cannot depend on treatment status. The propensity score is the coarsest balancing score (Lemma 6.1 below), and therefore provides the greatest dimension reduction without sacrificing identification.
Definition 6.2 (Propensity Score (Rosenbaum and Rubin 1983)) For a binary treatment \(T \in \{0,1\}\) and observed covariates \(X\), the propensity score is \[\pi(X) = P(T{=}1 \mid X).\]
In a randomized experiment, \(\pi(X)\) is a known constant for all units — equal to \(0.5\) under balanced Bernoulli assignment (\(P(T_i{=}1) = 0.5\) independently across units), and, marginally for each unit, under complete randomization with exactly \(n/2\) treated, where the assignment indicators are dependent because the treated count is fixed. In an observational study, \(\pi(X)\) varies with \(X\) and must be estimated from data.
Connection to strong ignorability. Under strong ignorability \((Y(0),Y(1)) \indep T \mid X\), the potential outcomes carry no further information about treatment assignment once \(X\) is given, so the treatment assignment mechanism satisfies \[P\bigl(T{=}1 \mid X,\, Y(0),\, Y(1)\bigr) = P(T{=}1 \mid X) = \pi(X).\] This equality is a consequence of the ignorability assumption, not part of the definition of \(\pi(X)\). What unconfoundedness rules out is precisely any residual dependence of treatment on the potential outcomes after conditioning on \(X\).
Theorem 6.1 (The Propensity Score Is a Balancing Score (Rosenbaum and Rubin 1983)) \(T \indep X \mid \pi(X)\).
Proof. It suffices to show \(P(T{=}1 \mid X, \pi(X)) = P(T{=}1 \mid \pi(X))\). Since \(\pi(X)\) is a deterministic function of \(X\), conditioning on both \(X\) and \(\pi(X)\) is the same as conditioning on \(X\) alone, so \(P(T{=}1 \mid X, \pi(X)) = P(T{=}1 \mid X) = \pi(X)\). For the right-hand side, the law of iterated expectations gives \[P\bigl(T{=}1 \mid \pi(X)\bigr) = \E\bigl[P(T{=}1 \mid X) \,\big|\, \pi(X)\bigr] = \E\bigl[\pi(X) \,\big|\, \pi(X)\bigr] = \pi(X),\] where the last equality holds because \(\pi(X)\) is measurable with respect to itself. Hence both sides equal \(\pi(X)\) and the conditional independence follows. \(\square\)
In the running example, the two cells with \(\pi = 0.5\) together contain \(200\) treated and \(200\) control units, and within this pooled stratum the covariate \(X\) is distributed identically in the two arms: half of the treated units and half of the controls come from each cell. That is the balancing property at work. It says nothing about outcomes: the two cells have different \(\mu(t,x)\), and pooling them is harmless for estimating a causal effect only because of the theorem of the next subsection.
6.2.2 The Propensity Score Theorem
The balancing property has a fundamental identification consequence: unconfoundedness at \(X\) implies unconfoundedness at any balancing score of \(X\), and in particular at the scalar \(\pi(X)\). Before stating that result we record a structural fact about balancing scores: among all of them, the propensity score is the coarsest. The identification theorem can then be proved at full generality, with the propensity score case obtained by taking \(b(X) = \pi(X)\).
Lemma 6.1 (The Propensity Score Is the Coarsest Balancing Score (Rosenbaum and Rubin 1983)) If \(b(X)\) is any balancing score — that is, \(T \indep X \mid b(X)\) — then \(\pi(X)\) is a function of \(b(X)\): \(\pi(X) = g(b(X))\) for some measurable function \(g\).
Proof. Since \(b(X)\) is a balancing score, \(T \indep X \mid b(X)\), which means \(P(T{=}1 \mid X,\, b(X)) = P(T{=}1 \mid b(X))\). But \(b(X)\) is a function of \(X\), so conditioning on \(b(X)\) is already implicit in conditioning on \(X\): \[\pi(X) = P(T{=}1 \mid X) = P\bigl(T{=}1 \mid X,\, b(X)\bigr) = P\bigl(T{=}1 \mid b(X)\bigr).\] Hence \(\pi(X) = P(T{=}1 \mid b(X))\), which is a measurable function of \(b(X)\) alone. \(\square\)
The lemma is the precise statement of the Rosenbaum–Rubin coarsest-balancing-score property: the propensity score is a measurable function of every balancing score, \(\pi(X) = g(b(X))\), or equivalently \(\sigma\{\pi(X)\} \subseteq \sigma\{b(X)\}\). Every balancing score is therefore at least as fine as \(\pi(X)\), which is what makes \(\pi(X)\) the coarsest balancing score and the maximally parsimonious adjustment variable. The reverse factorization need not hold: a general balancing score — \(X\) itself, for example — need not be a function of \(\pi(X)\). (By analogy with classical sufficiency, \(\pi(X)\) plays the role of a minimal sufficient reduction of \(X\) for the treatment indicator, although no parametric model is involved here.) With the lemma in hand, the identification theorem is straightforward.
Theorem 6.2 (Propensity Score Theorem (Rosenbaum and Rubin 1983)) Suppose joint conditional exchangeability holds: \((Y(0), Y(1)) \indep T \mid X\). Then for any balancing score \(b(X)\), \[\bigl(Y(0),\, Y(1)\bigr) \;\indep\; T \;\Big|\; b(X);\] no positivity is needed for this conditional-independence transfer. If, in addition, \(0 < \pi(X) < 1\) almost surely, then \(0 < P(T{=}1 \mid b(X)) < 1\) almost surely as well, so strong ignorability given \(X\) implies strong ignorability given \(b(X)\), and adjustment for \(b(X)\) identifies the potential-outcome means. In particular, taking \(b(X) = \pi(X)\) gives strong ignorability with the scalar propensity score in place of the full covariate vector.
6.2.3 Subclassification on the Propensity Score
The first estimator that the dimension reduction suggests is the stratified estimator of Chapter 5 with strata defined by the scalar \(\hat\pi(X)\) instead of by cells of \(X\). The score is a nuisance parameter, as the regression coefficient was in Chapter 5: fit \(\hat\pi(X_i)\), then stratify on it. Choose boundaries \(0 = b_0 < b_1 < \cdots < b_J = 1\) — quantiles of \(\hat\pi\) are the common choice — and let \(B_i(j) = \mathbb{I}\{b_{j-1} < \hat\pi(X_i) \le b_j\}\) indicate membership of unit \(i\) in stratum \(j\). Within stratum \(j\) compare arm means, then average the stratum contrasts with the stratum shares: \[\hat\tau(j) = \frac{\sum_{i=1}^n B_i(j)\,T_iY_i}{\sum_{i=1}^n B_i(j)\,T_i} - \frac{\sum_{i=1}^n B_i(j)(1-T_i)Y_i}{\sum_{i=1}^n B_i(j)(1-T_i)}, \qquad \hat\tau_{\mathrm{st}} = \sum_{j=1}^J \frac{n_j}{n}\,\hat\tau(j), \quad n_j = \sum_{i=1}^n B_i(j). \tag{6.1}\] The weights \(n_j/n\) target the ATE. Weighting the stratum contrasts by the treated shares instead, \[\hat\tau_{\mathrm{ATT,st}} = \sum_{j=1}^J \frac{n_{1j}}{n_1}\,\hat\tau(j), \qquad n_{1j} = \sum_i B_i(j)T_i,\quad n_1 = \sum_i T_i,\] targets the ATT. In the running example, taking the three score values as strata, the population version of Equation 6.1 is \(0.4 \times 2 + 0.4 \times 2 + 0.2 \times 4 = 2.4 = \tau_{\mathrm{ATE}}\), and the ATT version gives \((80 \times 2 + 200 \times 2 + 160 \times 4)/440 = 30/11 = \tau_{\mathrm{ATT}}\); the pooled stratum \(\pi = 0.5\) contributes its score-conditional effect \(2\), not either cell effect.
What it costs. With a continuous \(X\), the score is only approximately constant within a stratum, so some confounding remains even when \(\hat\pi\) is correct; more strata reduce it at the price of sparser cells and more variable \(\hat\tau(j)\). With \(J\) fixed, the residual bias does not vanish as \(n\) grows — a limitation of fixed-stratum subclassification, not of Theorem 6.2. For a single continuous confounder, under particular distributional models, Cochran (1968) found that \(J = 5\) strata of equal size remove about \(90\%\) of the bias of the unadjusted comparison, and extending his normal-model calculation to \(J = 10\) gives about \(96\%\); quintiles of \(\hat\pi\) are the common default. If either arm of a stratum is empty, \(\hat\tau(j)\) is undefined: revise the grouping or the estimator, and if units or strata are dropped, state the resulting target population, since the full-population ATE is no longer automatically the target.
Balance within strata is approximate. The true score satisfies \(T \indep X \mid \pi(X)\), but in practice one groups estimated scores into intervals \(B(X)\), and two things can leave imbalance within a stratum: estimation error in \(\hat\pi\), and variation of \(\pi\) inside the interval, since \(P\{T{=}1 \mid B(X)\} = \E\{\pi(X) \mid B(X)\} \ne \pi(X)\) unless \(\pi\) is constant on the interval. Within-stratum balance, \(\bar X_{1j} \approx \bar X_{0j}\) for every stratum \(j\), is therefore something to assess with the diagnostics of Section 6.3.5, not a theorem to invoke. Subclassification approximates adjustment on the score more and more closely as the strata become finer, provided both arms remain populated; matching (Section 6.4) compares units with nearby scores by a separate rule, possibly reusing controls, and its matched sets need not partition the sample, so it is not the limit of the stratification estimator.
Example 6.2 (Pooling Cells That Do Not Share a Score) Change one entry of the running example: let cell \((0,1)\) have \(N_{1x} = 150\) treated units instead of \(100\), so that \(\pi(0,1) = 0.75 \ne 0.5\). The cells \((1,0)\) and \((0,1)\) no longer share a score value and the theorem no longer licenses pooling them. Pooling them anyway gives treated shares \((100,150)/250 = (0.4, 0.6)\) and control shares \((100,50)/150 = (0.67, 0.33)\) from the two cells: the pooled stratum is not balanced. Its pooled contrast is \((100 \times 9 + 150 \times 7)/250 - 6 = 1.8\), while the average effect in the two cells is \((3+1)/2 = 2\), and the resulting ATE calculation becomes \(2.32\) instead of \(2.4\). (The ATT calculation happens to survive here, at \(125/49\), because both cells have control mean \(6\): imbalance and bias for a given estimand are distinct claims.)
6.3 Constructing the Propensity Score
The true propensity score \(\pi(X) = P(T{=}1 \mid X)\) is unknown in observational studies and must be estimated from the data. Its practical construction, however, requires more than choosing a binary-response algorithm. Three logically distinct decisions must be kept separate. First, causal reasoning determines the pretreatment adjustment set \(X\): the set must be sufficient to support conditional exchangeability, for example by blocking every back-door path from \(T\) to \(Y\) (Chapters 3 and 5). Second, statistical modeling determines how the treatment probability depends on the chosen covariates — via logistic regression, nonlinear terms, interactions, regularization, or flexible machine-learning methods. Third, design diagnostics determine whether the fitted score actually produces adequate balance and overlap for the chosen estimand. A good predictive model for treatment does not automatically produce a good causal design, and no statistical treatment model can repair an invalid adjustment set chosen from an incorrect causal graph.
6.3.1 Covariate Selection: What Belongs in \(X\)?
The propensity score inherits its causal validity from the adjustment set on which it is conditioned. The primary rule is therefore graphical and scientific rather than predictive: \(X\) should be a pretreatment set sufficient to block all back-door paths from \(T\) to \(Y\). The causal assumption is \(Y(t) \indep T \mid X\); estimating \(\pi(X) = P(T{=}1 \mid X)\) cannot make this assumption true if it was false for the chosen \(X\).
A bias–variance trade-off. Among the pretreatment variables that keep the adjustment set valid, the choice of what to put in the score model trades bias against variance. Too few covariates: omitting a confounder, or a term the score model needs (a square, an interaction), misspecifies the score for adjustment, and the resulting bias does not shrink as \(n\) grows. Too many covariates: a variable that is not needed for ignorability leaves a valid set valid, but it pushes the estimated scores toward zero or one, enlarges the weights, lowers the effective sample size (Section 6.3.5), and adds estimation noise. The two costs are not symmetric. The variance cost of an unnecessary covariate falls as \(n\) grows and is measured by the weight diagnostics. The bias of an omitted confounder does not fall with \(n\), and no outcome-blind diagnostic can size it: a balance check on the omitted variable shows that the arms differ on it, not whether that difference matters for \(Y\). When it is uncertain whether a pretreatment variable is a confounder, including it is therefore cheap insurance. The same reasoning applies to the functional form: richer terms in the score model cost variance and protect against misspecification bias, the lesson of Part 2 of the lab (Section 6.7.2). The insurance argument has two limits, taken up below: it covers only variables that keep the set valid, so mediators and colliders are excluded however uncertain one is; and a variable known to affect treatment only buys no protection and can amplify bias from unmeasured confounding.
Example 6.3 (Insurance against an Omitted Confounder) Let \(X_1, X_2\) be independent \(\mathrm{N}(0,1)\), \(T \sim \mathrm{Bernoulli}\{\expit(-0.3 + 0.6X_1 + 1.2X_2)\}\), and \(Y = T + X_1 + \gamma X_2 + \varepsilon\) with \(\varepsilon \sim \mathrm{N}(0,1)\), so that \(\tau_{\mathrm{ATE}} = 1\). The variable \(X_2\) strongly predicts treatment. With \(\gamma = 0\) it is unnecessary for adjustment; with \(\gamma = 0.3\) it is a weak confounder. Fit the logistic score by maximum likelihood on \(X_1\) alone or on \((X_1, X_2)\) and compute the Hájek ATE estimator; over \(2{,}000\) replications at \(n = 1{,}000\) (script covsel19.R, seed 19):
| Score model on | Bias (\(\gamma = 0\)) | SD (\(\gamma = 0\)) | Bias (\(\gamma = 0.3\)) | SD (\(\gamma = 0.3\)) |
|---|---|---|---|---|
| \(X_1\) only | \(-0.002\) | \(0.066\) | \(0.281\) | \(0.070\) |
| \(X_1\) and \(X_2\) | \(0.000\) | \(0.098\) | \(0.000\) | \(0.108\) |
With \(\gamma = 0\), including \(X_2\) costs variance: the SD rises from \(0.066\) to \(0.098\), and the control-arm effective sample size falls from about \(95\%\) to \(66\%\) of \(n_0\). With \(\gamma = 0.3\), omitting \(X_2\) costs a bias of \(0.281\), four times the SD, and the RMSE rises from \(0.108\) to \(0.289\). Here a wrong exclusion is far more expensive than a wrong inclusion, and only the cost of the inclusion is measured by the diagnostics.
Where each variable sits on this trade-off depends on its role. Within a causally valid pretreatment set, candidate variables need not play identical roles. A useful first taxonomy classifies them by whether they predict treatment, outcome, both, or neither (Brookhart et al. 2006; VanderWeele 2019). The roles themselves come from the causal graph; the two “predicts” columns describe tendencies, not definitions. In particular, a treatment predictor can predict the observed outcome through \(T\) without predicting the potential outcomes given a valid adjustment set, so predictive association with \(Y\) does not by itself make a variable a confounder.
| Variable type | Predicts \(T\)? | Predicts \(Y\)? | Typical recommendation |
|---|---|---|---|
| Confounder | yes | yes | include |
| Outcome predictor only | no or weakly | yes | usually include (balance it) |
| Treatment predictor only | yes | no | include only with caution |
| Predictor of neither | no | no | usually unnecessary |
Variables related to both treatment and outcome are central confounders and should ordinarily be included: they are needed to block back-door paths. Pretreatment variables that strongly predict the outcome may be useful even when they predict treatment only weakly, because they can improve precision; residual imbalance in a strong outcome predictor can matter greatly, so balance on such variables deserves priority in the diagnostics. Variables that strongly predict treatment but are unrelated to the potential outcomes require more caution: although they improve prediction of \(T\), they can push estimated scores toward zero or one, reduce overlap and effective sample size, and inflate the variance of weighted estimators without removing appreciable confounding bias; moreover, conditioning on such a near-instrument can amplify the bias arising from any residual unmeasured confounding. Pure noise variables mainly add estimation variability and may worsen finite-sample balance, especially with flexible learners.
These are tendencies rather than universal rules, and they do not justify automatic variable deletion. Whether a variable is truly unrelated to the potential outcomes is a scientific judgment subject to uncertainty, and plausible pretreatment confounders should generally be retained even when their treatment-model coefficients are small or statistically insignificant.
The adjustment set is validated as a whole, not variable by variable. For ordinary propensity-score adjustment aimed at a total effect, use a pretreatment set that blocks every back-door path, and do not adjust for mediators or for variables that open otherwise blocked noncausal paths. Post-treatment variables — variables affected by treatment — should not be conditioned on: doing so can block part of the causal effect, induce selection bias, and change the estimand. Pretreatment colliders can be equally damaging despite being measured before treatment: in the M-structure of panel (b) below, the variable \(C\) is pretreatment, yet conditioning on it opens the noncausal path \(T \leftarrow U_1 \rightarrow C \leftarrow U_2 \rightarrow Y\) that is blocked when \(C\) is left alone (Chapter 3). Whether a node acts as a collider is path-specific, and whether a set is a valid adjustment set depends on the whole graph and on the rest of the set; the conclusions drawn from the three panels are conclusions for those graphs. “Include every available variable” is in any case not a safe default.
6.3.2 Parametric Estimation by Logistic Regression
The classical approach models \[\log\frac{\pi(X)}{1 - \pi(X)} = \beta_0 + \sum_{j=1}^{p} \beta_j X_j + \sum_{j=1}^{p} \gamma_j\, h_j(X_j) + \sum_{j<k} \delta_{jk}\, X_j X_k,\] where the \(h_j\) are optional nonlinear transformations (polynomials, splines) and the cross-product terms capture interactions among covariates, and estimates the coefficients by maximum likelihood. The fitted values \(\hat\pi(X_i)\) are used in place of the true propensity score. Linearity on the logit scale in the raw covariates is an assumption, not a default truth: nonlinear terms and covariate-by-covariate interactions are often needed to achieve balance. (Treatment-by-covariate interactions have no place in this model — treatment is the response here.) Terms should be added based on causal knowledge and the balance diagnostics of Section 6.3.5, not predictive fit alone.
Variable selection by treatment-model \(p\)-values is generally inappropriate. A covariate may be an important confounder or outcome predictor even when its coefficient in the treatment model is small or statistically insignificant, and removing it can worsen causal balance without materially changing treatment-prediction accuracy. Backward selection driven by significance serves sparse prediction, which is not the objective of a propensity model. (Maximum likelihood is not the only way to fit the logistic model; Section 6.5.5 fits the same model so that the inverse-probability weights balance the covariates exactly.)
6.3.3 Flexible and Machine-Learning Estimators
When \(X\) is high-dimensional or when the true propensity score is non-linear, machine learning methods — gradient boosted trees, random forests, regularized logistic regression (LASSO, ridge) — can estimate \(\pi(X)\) more flexibly, reducing misspecification of the treatment model without fully prespecifying nonlinearities and interactions. Predictive accuracy alone, however, is not the relevant success criterion: hyperparameters tuned solely to classification loss (log loss, AUC) may improve prediction precisely by producing more extreme probability estimates, which destabilizes the resulting weights. A related issue is overfitting: an estimator that (nearly) perfectly separates treated and control units in-sample produces fitted scores near one for treated units and near zero for controls, mimicking an overlap violation that need not exist in the population. Cross-fitting (Chapter 12) removes the same-sample component of this problem by evaluating each unit’s score on a fold not used to fit it, but it does not create overlap and does not prevent extreme out-of-fold predictions: a genuinely near-deterministic treatment rule yields extreme scores on held-out data as well. Regularization, overlap diagnostics, and — when scientifically justified — trimming or re-targeting the estimand (developed later in this chapter) remain separate requirements. Whatever the learner, the fitted design must ultimately be evaluated through the balance and overlap it delivers; no machine-learning method eliminates the need for causal covariate selection or design diagnostics.
6.3.4 The Goal Is Balance, Not Treatment Prediction
A well-specified propensity model is still a model for \(P(T{=}1 \mid X)\), and accurate prediction is not itself a defect. It is, however, insufficient as a criterion. In ordinary classification, a model is rewarded for separating the two classes. In propensity-score analysis, strong separation signals trouble: fitted scores near one for treated units and near zero for controls mean little common support, extreme inverse-probability weights \(1/\hat\pi(X_i)\) and \(1/\{1-\hat\pi(X_i)\}\), and an estimator whose value depends on a handful of observations. A high AUC may therefore indicate a difficult causal comparison rather than a successful design. Note also that the AUC depends only on the ranking of the fitted probabilities — it is unchanged by any strictly increasing transformation of them — so it says nothing about how extreme the probabilities, and hence the weights, are. High AUC does not establish covariate balance, overlap, or causal validity; an additional treatment predictor can improve discrimination without improving confounding control. Treatment probabilities, balance, and weight stability must be evaluated separately, and a needed confounder is never dropped to make the arms look more alike.
The estimated score, moreover, is a design device, not the final diagnostic object. Conditioning on the true score balances \(X\) in distribution (Theorem 6.1), but matching or weighting approximately on an estimated score does not guarantee adequate finite-sample balance. After matching or weighting, balance must therefore be assessed on the original covariates and, when scientifically relevant, on nonlinear functions and interactions of them; similar distributions of \(\hat\pi(X)\) across arms do not by themselves establish that the underlying covariates are balanced.
6.3.5 Balance and Overlap Diagnostics
The standard scalar diagnostic for covariate balance is the standardized mean difference (SMD). For covariate \(X_j\), let \(\bar X_{1j}\) and \(\bar X_{0j}\) denote the treated and control sample means and \(s_{1j}^2\) and \(s_{0j}^2\) the corresponding sample variances. Then \[\mathrm{SMD}(X_j) = \frac{\bar X_{1j} - \bar X_{0j}}{\sqrt{\bigl(s_{1j}^2 + s_{0j}^2\bigr)/2}};\] for a binary covariate, the numerator is the difference in proportions, with the analogous pooled scale in the denominator. Dividing by the pooled scale makes the diagnostic free of the units of \(X_j\) and — unlike a \(t\)-statistic — insensitive to the sample size, so it measures the magnitude of imbalance rather than its statistical detectability. A common informal benchmark treats \(|\mathrm{SMD}| \le 0.1\) as adequate balance; this is a heuristic, not a test, and whether residual imbalance is consequential depends on how strongly the covariate predicts the outcome, on the sample size, and on the estimand. An SMD below the benchmark means the means are approximately balanced, not equal. For weighted or matched analyses, the arm means are replaced by their design-weighted counterparts (with weights given by the scheme in use, and reused control units counted with their matching multiplicities); the denominator, however, should be held fixed at the pre-adjustment pooled scale, so that before-and-after values are directly comparable.
Balance of means alone can be inadequate when the outcome depends nonlinearly on the covariates. Useful supplements include variance ratios, quantile differences, empirical-distribution plots, and SMDs computed on squares, interactions, and other scientifically relevant transformations of the covariates.
Balance and overlap are distinct, and both must be checked: a design can show good mean balance after weighting while relying on a few extreme weights, and raw overlap can look reasonable while nonlinear imbalance remains. Overlap diagnostics start with plots of the estimated score distributions in the two arms on a common scale. Which comparison matters depends on the estimand: for the ATT, the question is whether the control scores cover the treated-score distribution; for the ATE, both arms must cover the full target population. Weight summaries — minimum, maximum, selected quantiles, and the coefficient of variation — expose instability directly. For matching, the bookkeeping of the design plays the same role: the number of unmatched treated units, the number of distinct controls used and how often each is reused, and the distribution of match distances.
The cost of extreme weights can be summarized by the effective sample size of a weighted arm with nonnegative weights \(\{w_i\}\), \[n_{\mathrm{eff}} = \frac{\bigl(\sum_i w_i\bigr)^2}{\sum_i w_i^2},\] which equals the actual sample size when the weights are constant and collapses toward \(1\) when a single weight dominates. For two-arm estimators it is usually most informative arm by arm: \(n_{\mathrm{eff},t} = \bigl(\sum_{i: T_i = t} w_i\bigr)^2 / \sum_{i: T_i = t} w_i^2\). Its interpretation as a sample size is exact under a simple model: for independent outcomes with common variance \(\sigma^2\) and weights treated as fixed, \(\mathrm{Var}\bigl(\sum_i w_iY_i/\sum_i w_i \mid w\bigr) = \sigma^2/n_{\mathrm{eff}}\), so a weighted arm with \(500\) units but \(n_{\mathrm{eff}} = 40\) behaves like \(40\) equally weighted observations. Heterogeneous variances, varying conditional means, estimated weights, and reused controls all complicate this reading; in general \(n_{\mathrm{eff}}\) summarizes the concentration of the weights and is a diagnostic rather than the estimator’s actual precision. Reporting it alongside the point estimate makes the price of near-overlap violations concrete.
6.3.6 Design Objectives Depend on the Estimand
The same propensity model need not serve every estimator equally, so the diagnostics above should be tailored to the target. For matching (developed in Section 6.4), the score must support close matches and covariate balance within the matched sample; the relevant diagnostics are match distances, residual SMDs among matched units, unmatched treated units, and control reuse. For ATE weighting, with weights \(1/\hat\pi(X_i)\) for treated and \(1/\{1-\hat\pi(X_i)\}\) for control units, both arms are re-weighted to the full target population, so weighted balance in both arms, weight tails, and arm-specific effective sample sizes are the primary concerns. For ATT weighting, with weights \[w_i = T_i + (1 - T_i)\,\frac{\hat\pi(X_i)}{1 - \hat\pi(X_i)},\] the treated arm is left unweighted and the central question is whether the controls can represent the treated units, so control coverage of the treated-score distribution is the key diagnostic. (These are the weights whose population versions appear in the third identity of Lemma 6.4 and in the lab’s HJ-ATT estimator.)
The figure below displays the four weighting schemes as functions of the propensity score. The picture makes estimand targeting immediate: each scheme up-weights the units that must stand in for the population it targets, and instability appears exactly where a required weight diverges.
The general principle: a propensity model cannot be evaluated independently of the estimand and the estimator that will use it.
6.3.7 A Design-Stage Workflow
Propensity-score design is typically iterative: specify a treatment model, construct the proposed matched or weighted sample, assess balance and overlap, revise the design when the diagnostics are inadequate, and repeat. Revisions may include nonlinear terms, interactions, an alternative link or learner, regularization, a modified matching ratio or caliper, a different weighting target, or trimming. The outcome data remain hidden throughout this cycle.
Outcome blinding, in particular, does not mean that the specification is fixed after one fit. Three levels should be distinguished. Outcome-blind design: no outcome data are used to choose the treatment model or the design — not merely that no effect estimate is inspected; prior scientific knowledge about which covariates predict the outcome is compatible with this principle and should inform the adjustment set. Balance-driven revision: altering the model in response to observed imbalance is legitimate — indeed expected. Design lock: once balance and overlap are judged acceptable, the matching or weighting rule is fixed, and only then are outcomes analyzed. The benefit is not that subjectivity disappears; it is that design decisions cannot be selected because they produce a preferred causal estimate.
Example 6.4 (Variable Selection: Higher AUC, Worse Design) Consider five candidate variables for a treatment–outcome study: age \(A\) and baseline severity \(S\), each affecting both treatment and outcome; hospital preference \(H\), strongly predictive of treatment but essentially unrelated to the potential outcomes; post-treatment adherence \(M\); and a pretreatment collider \(C\) driven by two latent variables as in panel (b) of the covariate-role figure. Compare three propensity models: \[\text{Model 1: } \pi(A, S), \qquad \text{Model 2: } \pi(A, S, H), \qquad \text{Model 3: } \pi(A, S, H, M, C).\] Model 1 contains the essential adjustment set. Model 2 predicts treatment better but, if \(H\) nearly determines treatment, pushes fitted scores toward the boundaries, inflating weights and shrinking the effective sample size without removing additional confounding. Model 3 is causally invalid regardless of its fit: it conditions on the mediator \(M\) and the collider \(C\). Qualitatively, as an illustrative scenario rather than a general ordering:
| Model | AUC | Max weight | \(n_{\mathrm{eff}}\) | Causally valid? |
|---|---|---|---|---|
| \(\pi(A,S)\) | moderate | moderate | high | yes |
| \(\pi(A,S,H)\) | high | large | low | yes, but imprecise |
| \(\pi(A,S,H,M,C)\) | very high | extreme | poor | no |
The model with the highest AUC is the worst causal design, for two different reasons: \(H\) makes it imprecise, \(M\) and \(C\) make it invalid.
Example 6.5 (A Case Study of the Workflow: Prenatal Barbiturate Exposure) Imbens and Rubin (2015), Chapter 13, illustrate the design stage with the study of Reinisch et al. (1995) on in utero exposure to barbiturates and adult cognitive outcomes: \(7{,}943\) Copenhagen births, of which \(745\) were exposed, with seventeen pretreatment covariates on the mother and the pregnancy. The exposed group differs systematically from the unexposed: higher socioeconomic status, older mothers, more pregnancy complications. The propensity model is built in three steps. (1) A substantive set — the child’s sex, the mother’s age, socioeconomic status — enters the logistic model regardless of its association with exposure. (2) Linear terms for the remaining covariates are proposed one at a time by their likelihood-ratio contribution to the treatment model, while that contribution exceeds a threshold. (3) Second-order terms, squares and interactions, are proposed by the same rule with a lower threshold. The sample is then subclassified on \(\hat\pi\) and covariate balance is assessed within strata, as in Section 6.2.3.
This procedure is consistent with the warning against selecting variables by treatment-model \(p\)-values, and it is worth seeing why. Step 1 fixes the confounders on causal grounds, and they are retained whatever their coefficients. Steps 2 and 3 propose additional terms by treatment-model fit; the design is then judged by the covariate balance it produces, not by the fit. Nothing substantive is dropped, and likelihood-ratio screening does not itself guarantee balance — the balance assessment is a separate step that can send the analyst back to the model. The warning is against dropping a plausible confounder because its coefficient is small; it is not against adding terms until the strata are balanced, which is exactly the balance-driven revision of the workflow.
6.4 Matching on the Propensity Score
The propensity score also motivates matching: for each treated unit \(i\), find a control unit \(j\) with \(\hat\pi(X_j) \approx \hat\pi(X_i)\), and use \(Y_j\) as a local estimate of \(\E\bigl[Y(0) \mid \pi(X) = \pi(X_i)\bigr]\) (Abadie and Imbens 2006, 2016). It is important to be precise about what is being estimated: the matched control outcome \(Y_j\) is not a substitute for the individual counterfactual \(Y_i(0)\), which is unobservable and contains an irreducible idiosyncratic component. Rather, \(Y_j\) is a noisy local approximation to the conditional mean \(\E[Y(0) \mid \pi(X) = \pi(X_i)]\), which is identified from observed control outcomes via the chain \[\E[Y(0) \mid \pi(X) = \pi(X_i)] \;=\; \E[Y(0) \mid \pi(X) = \pi(X_i),\, T = 0] \;=\; \E[Y \mid \pi(X) = \pi(X_i),\, T = 0],\] where the first equality uses Theorem 6.2 (\(Y(0) \indep T \mid \pi(X)\) under strong ignorability) and the second uses consistency (\(Y = Y(0)\) when \(T = 0\)); positivity on the control side makes the conditional mean well defined. Averaging across treated units targets — and, under the regularity conditions discussed below, consistently estimates — the ATT: \[\hat\tau_{\mathrm{ATT}} = \frac{1}{n_1}\sum_{i:\, T_i=1} \Bigl[Y_i - Y_{\hat\jmath(i)}\Bigr],\] where \(n_1 = \sum_{i=1}^n T_i\) is the number of treated units and \(\hat\jmath(i)\) is the index of the matched control unit. This is the third implementation of one identification formula: stratify (Section 6.2.3), weight (Section 6.5), match.
6.4.1 One-to-One Nearest Neighbor Matching
The most common matching algorithm is nearest-neighbor (1:1) matching (Abadie and Imbens 2006): for each treated unit \(i\), find \[\hat\jmath(i) = \arg\min_{j:\, T_j=0} \bigl|\hat\pi(X_i) - \hat\pi(X_j)\bigr|.\] When several controls are equally close, a convention is required — deterministic tie-breaking (e.g., the smallest index), averaging the outcomes of all equally close controls, or random selection; the choice affects the exact estimator and its variance, though not the identification logic.
Matched control units may be selected with or without replacement. With replacement, every treated unit receives its closest control, so matches are closer; the price is that a few controls may be reused many times, which reduces the effective amount of control information. Without replacement, each control is used once, the procedure needs at least as many available controls as retained treated units, and match quality degrades as the pool empties. Smaller match distances do not by themselves imply a universal ordering of bias or variance between the two schemes.
Matching with replacement is weighting with integer weights. If \(K_j\) denotes the number of treated units matched to control \(j\), then \[\hat\tau_{\mathrm{NNM}} = \frac{1}{n_1}\sum_{i:\, T_i=1} Y_i - \frac{1}{n_1}\sum_{j:\, T_j=0} K_j\, Y_j, \tag{6.2}\] so the control arm is reweighted by \(K_j/n_1\), a data-dependent analogue of the ATT odds weights of Section 6.3.6. The identity makes control reuse visible — a control with large \(K_j\) carries a large weight — and is the starting point for the variance theory cited below.
6.4.2 Caliper Matching
Caliper matching restricts matches to pairs within a maximum distance \(\delta\) (the “caliper”) on the propensity score scale: \[\hat\jmath(i) = \arg\min_{j:\, T_j=0,\; |\hat\pi(X_i) - \hat\pi(X_j)| \le \delta} \bigl|\hat\pi(X_i) - \hat\pi(X_j)\bigr|.\] Treated units with no control match within the caliper are discarded. The caliper may be defined on the probability scale, as written here, or — commonly in practice — on the logit scale \(\mathrm{logit}\,\hat\pi(X)\), which spaces units more evenly near the boundaries of \([0,1]\); the chosen scale must be reported, since numerical caliper values are not comparable across scales. Because matchable treated units are retained and the rest discarded, the procedure no longer targets the original ATT: it targets an ATT restricted to the retained treated subpopulation with adequate overlap. Dropping treated units changes the finite-sample target, and the population version of that target depends on the limiting retention rule, the caliper, and the control support, not only on the convergence of \(\hat\pi\); under adequate overlap a fixed caliper eventually retains nearly all treated units.
6.4.3 The Prognostic Score: The Outcome-Side Mirror
The propensity score reduces \(X\) to what bears on treatment. Hansen (2008) proposed the mirror image: reduce \(X\) to what bears on the untreated outcome. The ignorability condition \(Y(0) \indep T \mid X\) is symmetric in \(T\) and \(Y(0)\), and the theory of Section 6.2 carries over with the two exchanged.
Definition 6.3 (Prognostic Score (Hansen 2008)) A function \(\Psi(X)\) is called a prognostic score if \[Y(0) \indep X \mid \Psi(X).\]
As with balancing scores, \(X\) itself is trivially a prognostic score. Hansen’s examples: under a location model \(Y(0) = m_0(X) + \varepsilon\) with \(\varepsilon \indep X\), the scalar \(m_0(X) = \E\{Y(0) \mid X\}\) is a prognostic score; under a generalized linear model for \(Y(0)\) given \(X\), so is the linear predictor; and when the variance of \(Y(0)\) also depends on \(X\), the mean and variance functions together form a two-dimensional prognostic score. The definition is stated for \(Y(0)\) because \(Y(0)\) is observed in the control arm, from which a prognostic score is estimated, and because \(Y(0)\) is what the ATT must impute.
Lemma 6.2 (The Conditional Law of \(Y(0)\) Is the Coarsest Prognostic Score) Let \(F_0(y \mid X) = P\{Y(0) \le y \mid X\}\). Then (i) \(F_0(\cdot \mid X)\) is a prognostic score; and (ii) if \(\Psi(X)\) is any prognostic score, \(F_0(\cdot \mid X)\) is a function of \(\Psi(X)\).
Part (i) is Theorem 6.1 with \(Y(0)\) in place of \(T\), and part (ii) is Lemma 6.1: the mirror of \(\pi(X) = P(T{=}1 \mid X)\) is the whole conditional law of \(Y(0)\). For a binary \(T\) the conditional law is its mean, which is why the coarsest balancing score is a scalar; for a general \(Y(0)\) the law needs more than its mean. Under the location model, \(F_0(y \mid X) = P(\varepsilon \le y - m_0(X))\) is a function of \(m_0(X)\), so \(m_0(X)\) is the coarsest prognostic score. Without such a model \(m_0(X)\) is in general not a prognostic score, but it still suffices for mean contrasts, as part (iii) of the next theorem shows.
Theorem 6.3 (Prognostic Score Theorem) Suppose \(Y(0) \indep T \mid X\).
For any prognostic score \(\Psi(X)\), \(Y(0) \indep T \mid \Psi(X)\).
If, in addition, consistency holds, \(\E|Y(0)| < \infty\), and \(P\{T{=}1 \mid \Psi(X)\} < 1\) almost surely, then \[\tau_{\mathrm{ATT}} = \E\Bigl[\E\{Y \mid T{=}1, \Psi(X)\} - \E\{Y \mid T{=}0, \Psi(X)\} \Bigm| T{=}1\Bigr].\]
The conclusion of (ii) holds with \(\Psi(X)\) replaced by \(m_0(X) = \E\{Y(0) \mid X\}\), provided \(P\{T{=}1 \mid m_0(X)\} < 1\) almost surely, whether or not \(m_0(X)\) is a prognostic score.
Parts (i) and (ii) are Propositions 3 and 4 of Hansen (2008). Part (ii) is the ATT formula of Corollary 6.1 with \(\Psi(X)\) in place of \(X\), and part (iii) explains why the proof of that corollary could take \(h = m_0(X)\): for a mean target, the location-model score is enough.
Where the mirror breaks. The first four rows of Table 6.1 are exact mirror images; the last four are where the exchange of \(T\) and \(Y(0)\) fails, and each failure has one source: \(T\) is binary and observed for every unit, while \(Y(0)\) is general and observed only for controls. Positivity is the most consequential. For a balancing score, \(P\{T{=}1 \mid b(X)\} = \pi(X)\) because \(\pi\) is a function of \(b\), so positivity in \(b(X)\) and in \(X\) are equivalent. For a prognostic score, \(P\{T{=}1 \mid \Psi(X)\} = \E\{\pi(X) \mid \Psi(X)\}\) only averages \(\pi\), so the overlap condition of part (ii) can hold although \(\pi(x) = 1\) on part of the treated support, where the ATT is not identified by adjustment for \(X\) (Section 6.6.4). The gain is bought with outcome knowledge: at such \(x\) no control is observed, so the value of \(\Psi(x)\) comes from a model fitted to controls elsewhere and extrapolated. The model, not the data, supplies that part of the answer, as with any parametric extrapolation.
| Balancing side | Prognostic side | |
|---|---|---|
| The mirror: exchange \(T\) and \(Y(0)\) | ||
| Definition | \(T \indep X \mid b(X)\) (Definition 6.1) | \(Y(0) \indep X \mid \Psi(X)\) (Definition 6.3) |
| Canonical score | \(\pi(X) = P(T{=}1 \mid X)\) (Theorem 6.1) | \(F_0(\cdot \mid X) = P\{Y(0) \le \cdot \mid X\}\) (Lemma 6.2) |
| Coarsest | \(\pi\) is a function of every \(b\) (Lemma 6.1) | \(F_0(\cdot \mid X)\) is a function of every \(\Psi\) (Lemma 6.2) |
| Ignorability transfers | \((Y(0), Y(1)) \indep T \mid b(X)\) (Theorem 6.2) | \(Y(0) \indep T \mid \Psi(X)\) (Theorem 6.3) |
| Where the mirror breaks | ||
| Size of the score | a scalar: the law of a binary \(T\) is its mean | a distribution; \(m_0(X)\) under a location model, and \(m_0\) suffices for means |
| Positivity transfers | both ways: \(P\{T{=}1 \mid b(X)\} = \pi(X)\) | one way: \(P\{T{=}1 \mid \Psi(X)\} = \E\{\pi(X) \mid \Psi(X)\}\); \(\Psi\) is extrapolated where \(\pi = 1\) |
| Outcomes covered | both: ATE, ATT, and ATC from one score | \(Y(0)\) only: the ATT; the ATE needs a score for \(Y(1)\) too |
| Fitting and checks | all units, outcome-blind | controls only, using outcomes |
6.4.4 Matching under a Misspecified Score
Weighting and matching use a fitted score in different ways. IPW uses its numerical values: the weight is \(1/\tilde\pi\) or \(\tilde\pi/(1-\tilde\pi)\), and wrong values generally give wrong weights and a wrong probability limit. There are exceptions. If the fitted logit differs from the true one by a constant, \(\tilde\pi(X) = \expit\{\mathrm{logit}\,\pi(X) + c\}\), then every control odds weight is multiplied by \(\exp(c)\): \[\frac{\tilde\pi(X)}{1-\tilde\pi(X)} = \exp(c)\,\frac{\pi(X)}{1-\pi(X)},\] and the common factor cancels in the Hájek ATT estimator, which is then exactly unchanged. Matching uses the score to decide which units are alike. The next lemma makes the identification part of this precise; write \(\tilde b(X)\) for the limit of a fitted score, correct or not.
Lemma 6.3 (When Matching on a Fitted Score Is Valid) Suppose \(Y(0) \indep T \mid X\), and let \(\tilde b(X)\) be any function of \(X\). If \(\pi(X)\) is a function of \(\tilde b(X)\), or if some prognostic score is a function of \(\tilde b(X)\), then \(Y(0) \indep T \mid \tilde b(X)\).
The lemma is an identification statement. Consistency of the matching estimator of the ATT needs more: overlap, smoothness of the conditional mean of \(Y(0)\) in the score, match discrepancies that shrink as \(n\) grows, and moment conditions. With these, the lemma gives the condition of Lesson 7 of the lab. The conditions of the lemma involve \(\tilde b\) only through its level sets, the sets of units it treats as alike: if they hold for \(\tilde b\), they hold for \(g(\tilde b)\) for every one-to-one \(g\). A misspecification that distorts the values of the score but keeps its level sets — a wrong link, a miscalibrated fit, a monotone distortion of the fitted probabilities — therefore leaves the identifying condition intact. It generally biases IPW, while matching on the fitted score can remain consistent.
The nearest-neighbor algorithm is not invariant in the same way, because it uses distances, and a one-to-one transformation can change them enough to change the match. Take a treated unit with score \(0.50\) and two controls with scores \(0.40\) and \(0.59\). The distances are \(0.10\) and \(0.09\), so the match is \(0.59\). Square all scores, a one-to-one map on \([0,1]\): the distances become \(0.0900\) and \(0.0981\), and the match becomes \(0.40\). A fixed caliper is scale dependent for the same reason (Section 6.4.2). A transformation that keeps the level sets changes which matches are made in a finite sample, not the identifying condition; stratification on quantiles of the score is exactly invariant.
Misspecifications that keep the level sets are common. If \(\pi(X) = F(\beta^\top X)\) for an increasing link \(F\) and \(X\) is Gaussian, or more generally elliptically symmetric, the logistic maximum-likelihood fit converges to \(\expit(a + c\beta^\top X)\) for some constants \(a\) and \(c\), with \(c > 0\) unless the fit is degenerate (Li and Duan 1989): the fitted score is then a monotone function of the true one, whatever \(F\) is. Waernbaum (2012) studies such robustness classes for matching and compares them with doubly robust estimation.
The protection has limits. A misspecification that merges units with different true scores and different prognoses satisfies neither condition of the lemma, and then matching is biased along with IPW. Matching is more robust than IPW in a specific sense: its identifying condition survives a larger class of misspecifications, not all of them, and consistency still needs the regularity conditions above.
Example 6.6 (Two Misspecifications) Let \(X\) have three independent \(\mathrm{N}(0,1)\) components and \(\tau(X) = 1 + 0.5X_1\). In design A the true score has a complementary log-log link, \(\pi(X) = 1 - \exp[-\exp\{-0.8 + 0.6X_1 + 0.4X_2 - 0.3X_3\}]\), and \(m_0(X) = X_1^2 + X_2 + \exp(0.5X_3)\); the logistic fit has the wrong link but the right index. In design B the true score is U-shaped, \(\pi(X) = \expit(-1 + 0.9X_1^2 + 0.3X_2)\), and \(m_0(X) = X_1^2 + X_2\); the linear logistic fit merges units with \(X_1\) and \(-X_1\), which differ in score and in prognosis. With \(n = 2{,}000\) and \(1{,}000\) replications (script robmatch.R, seed 6), the biases (SDs) for the ATT are:
| True score | Logistic fit | |||
|---|---|---|---|---|
| Design | Hájek ATT | NNM | Hájek ATT | NNM |
| A: wrong link | \(0.012\) \((0.246)\) | \(0.034\) \((0.164)\) | \(0.177\) \((0.110)\) | \(0.037\) \((0.155)\) |
| B: merged level sets | \(0.100\) \((0.481)\) | \(0.111\) \((0.144)\) | \(1.095\) \((0.083)\) | \(1.089\) \((0.097)\) |
In design A, \(X\) is Gaussian and the true score is an increasing function of the index \(0.6X_1 + 0.4X_2 - 0.3X_3\), so by Li and Duan (1989) the limit of the logistic fit is an increasing function of the true score: in the population the fit keeps the level sets exactly, and the first condition of the lemma holds. (The finite-sample rank correlation between the fitted and the true score is \(0.9999\).) Matching on the fit behaves like matching on the true score. The fitted logit is not the true logit plus a constant, so the exception above does not apply, and the Hájek estimator is biased here by \(0.177\), about \(50\) Monte Carlo standard errors. In design B the rank correlation is \(0.41\) and both estimators fail. (The true-score rows of design B carry some finite-sample bias from thin overlap at large \(|X_1|\); the contrast is \(0.1\) against \(1.1\).) The table illustrates the lemma; it does not prove consistency, which rests on the lemma and the regularity conditions.
Part 2 of the lab (Section 6.7.2) lies between the two designs, and the second condition of the lemma offers an explanation after one reduction. In the lab the five covariates are independent \(\mathrm{N}(0,1)\) and the error is independent of them; in Part 2, \(X_2\) and \(X_3\) affect only the outcome. Let \(Z = (X_1, X_4, X_5)\). Treatment depends on \(X\) only through \(Z\), so \(Y(0) \indep T \mid Z\) and \(Z\) is a sufficient adjustment set. Given \(Z\), the outcome-only terms \(0.5X_2 + 0.4X_3\) join \(\varepsilon\) in a residual that is independent of \((Z, T)\), so \[Y(0) = 8 + r(Z) + (0.5X_2 + 0.4X_3 + \varepsilon), \qquad r(Z) = 0.8X_1 + 0.3X_4 + 0.2X_5,\] and \(r(Z)\) is a prognostic score relative to \(Z\): \(Y(0) \indep Z \mid r(Z)\). (It is not a prognostic score relative to \(X\), because the law of \(Y(0)\) given \(r\) still depends on \(X_2\) and \(X_3\).) Applied with \(Z\) in place of \(X\), the lemma covers any fitted score that determines \(r(Z)\), such as an increasing function of \(r(Z)\). The misspecified linear fit does not keep the level sets of the true score (rank correlation \(0.83\)), but its index, approximately \(0.64X_1 + 0.19X_4 + 0.10X_5\) with negligible coefficients on \(X_2\) and \(X_3\), has correlation \(0.994\) with \(r(Z)\): the fitted score is nearly a prognostic score for \(Z\). This helps explain why NNM has a small bias in this simulation (\(0.020\), against \(0.154\) for the Hájek ATT). A correlation of \(0.994\) is not an exact robustness guarantee and gives no bound on the bias.
6.5 Inverse Probability Weighting
6.5.1 The IPW Identification Formula
The ATE admits an alternative representation as an inverse probability weighting (IPW) formula, in which each observed outcome is weighted by the inverse probability of the treatment actually received. The survey analogy is exact: the treated arm is a sample from the population of all \(n\) units with known inclusion probabilities \(\pi(X_i)\), and the Horvitz–Thompson estimator is the natural tool. The precise statement, arm by arm:
Theorem 6.4 (IPW Identification) Fix \(t \in \{0, 1\}\) and assume: (i) consistency: \(T = t\) implies \(Y = Y(t)\); (ii) weak conditional exchangeability: \(Y(t) \indep T \mid X\); (iii) positivity: \(P(T{=}t \mid X) > 0\) almost surely; and (iv) integrability: \(\E|Y(t)| < \infty\). Then \[\E\!\left[\frac{T \cdot Y}{\pi(X)}\right] = \E[Y(1)] \;\;\text{for } t = 1, \qquad \E\!\left[\frac{(1-T) \cdot Y}{1 - \pi(X)}\right] = \E[Y(0)] \;\;\text{for } t = 0.\] If the conditions hold for both treatment levels, \[\tau_{\mathrm{ATE}} = \E\!\left[\frac{T \cdot Y}{\pi(X)} - \frac{(1-T) \cdot Y}{1 - \pi(X)}\right].\]
In the running example, the treated arm of cell \((0,0)\) has \(80\) units and weight \(1/0.2 = 5\), so it stands in for all \(400\) units of that cell; the \(160\) treated units of cell \((1,1)\) have weight \(1.25\) and stand in for \(200\). Each reweighted arm reproduces the population cell counts, and then \(\E\{TY/\pi(X)\} = \sum_x P(x)\,\pi(x)\,\mu(1,x)/\pi(x) = \sum_x P(x)\mu(1,x) = 7.6\), \(\E\{Y(0)\} = 5.2\), and \(\tau_{\mathrm{ATE}} = 2.4\). The idealized table has \(N_{1x} = N_x\pi(x)\) exactly; with random assignment the weighted counts match the population counts in expectation, which is the content of Lemma 6.4 below.
6.5.2 Why the Weights Work: Weighted-Balance Identities
The balancing theorem and the IPW formula are two faces of the same mechanism, and the connection can be made algebraic: inverse-probability weights re-create, inside each treatment arm, the covariate distribution of the target population.
Lemma 6.4 (Weighted-Balance Identities) Let \(h(X)\) be any integrable function of the covariates and suppose \(0 < \pi(X) < 1\) almost surely. Then \[\E\!\left[\frac{T\, h(X)}{\pi(X)}\right] = \E[h(X)] \qquad\text{and}\qquad \E\!\left[\frac{(1-T)\, h(X)}{1-\pi(X)}\right] = \E[h(X)].\] Moreover, for the treated target population, \[\frac{1}{P(T{=}1)}\, \E\!\left[(1-T)\,\frac{\pi(X)}{1-\pi(X)}\, h(X)\right] = \E\bigl[h(X) \mid T{=}1\bigr].\]
Taking \(h \equiv 1\) shows that each arm’s inverse-weight total, divided by the full sample size \(n\), has expectation one — the weight-sum quantities \(w_T\) and \(w_C\) of the next subsection. Taking \(h(X) = X_j\) shows that the weighted treated and control samples reproduce the covariate means of the target population: the full population under the ATE weights \(1/\pi(X)\) and \(1/\{1-\pi(X)\}\), the treated subpopulation under the ATT odds weights \(\pi(X)/\{1-\pi(X)\}\) applied to the control arm. This is why the weights take their particular forms. Note that no potential outcomes appear in the lemma: like the balancing theorem, it is a property of the assignment mechanism alone. Theorem 6.4 is the step from \(h(X)\) to \(Y(1)\): the target is \(\E\{Y(1)\}\), not \(\E(Y)\); ignorability turns \(\E\{TY(1)/\pi(X)\}\) into \(\E[\pi(X)\E\{Y(1)\mid X\}/\pi(X)]\), and consistency replaces \(TY(1)\) by the observed \(TY\).
Dividing the third identity by its \(h \equiv 1\) case, \(\E\{(1-T)\,o(X)\} = P(T{=}1)\) with \(o(X) = \pi(X)/\{1-\pi(X)\}\), gives the normalized balance identity \[\frac{\E\{o(X)\,h(X) \mid T{=}0\}}{\E\{o(X) \mid T{=}0\}} = \E\{h(X) \mid T{=}1\} \qquad\text{for every integrable } h(X): \tag{6.3}\] ideal odds weighting balances every function of \(X\) between the weighted controls and the treated population — the covariates, their squares and products, and any prognostic function such as \(m_0(X) = \E\{Y(0) \mid X\}\) or \(P(Y(0) > t \mid X)\) (Section 6.4.3) — and no causal assumption is involved. The left side is the Hájek form. Equation 6.3 is a population identity under the true odds, not a sample guarantee: a fitted score balances none of these functions exactly in a realized sample, and no sample can balance all of them. Which functions to balance is a design choice, which Section 6.5.5 makes by calibration, with prognostic functions as the natural candidates.
The one choice of \(h\) that needs ignorability is \(h = Y(0)\), and it identifies the ATT.
Corollary 6.1 (IPW Identification of the ATT) Assume consistency, \(Y(0) \indep T \mid X\), control-arm positivity \(\pi(X) < 1\) almost surely, and \(\E|Y(0)| < \infty\). Then \[\E\{Y(0) \mid T{=}1\} = \frac{\E\{o(X)\,Y \mid T{=}0\}}{\E\{o(X) \mid T{=}0\}}, \qquad \tau_{\mathrm{ATT}} = \E(Y \mid T{=}1) - \frac{\E\{o(X)\,Y \mid T{=}0\}}{\E\{o(X) \mid T{=}0\}}.\]
Ignorability enters only to replace \(m_0(X)\) by arm-specific conditional means; the balancing itself is Equation 6.3. The right side of the first display is the population version of the Hájek-ATT estimator of Section 6.5.3.
Compared with the ATE, one arm is reweighted rather than two, the assumptions are one-sided — \(Y(0) \indep T \mid X\) and \(\pi(X) < 1\), so the ATT survives regions where \(\pi(x) = 0\) (Section 6.6.4) — and the target population is \(X \mid T{=}1\), whose law has density \(\pi(x)f_X(x)/P(T{=}1)\). In the running example, the odds \((0.25, 1, 1, 4)\) applied to the control counts \((320, 100, 100, 40)\) give weighted counts \((80, 100, 100, 160) = N_{1x}\), so \(\E\{Y(0) \mid T{=}1\} = 2720/440 = 68/11\), \(\E(Y \mid T{=}1) = 98/11\), and \(\tau_{\mathrm{ATT}} = 30/11\). The ATC is the mirror image.
6.5.3 The Horvitz–Thompson and Hájek Estimators
Write \(\hat\pi_i = \hat\pi(X_i)\) and define the inverse-probability weights \(a_i = T_i/\hat\pi_i\) and \(c_i = (1-T_i)/(1-\hat\pi_i)\). The sample analogue of Theorem 6.4 is the Horvitz–Thompson (HT) IPW estimator: \[\hat\tau_{\mathrm{HT}} = \frac{1}{n}\sum_{i=1}^n (a_i - c_i)\,Y_i = \frac{1}{n}\sum_{i=1}^n \left[\frac{T_i Y_i}{\hat\pi(X_i)} - \frac{(1-T_i) Y_i}{1-\hat\pi(X_i)}\right].\] This estimator is consistent when \(\hat\pi(X)\) converges to \(\pi(X)\), given positivity and mild moment conditions — with flexible nuisance estimators evaluated on the same sample, additional conditions such as sample splitting may be required (Chapter 12) — but it can be inefficient due to extreme weights. In practice, a fitted score used for weighting is evaluated through weighted covariate balance, the weight distribution, and effective sample sizes (Section 6.3.5 and Section 6.3.7). The normalized (Hájek) estimator divides by the sum of weights within each group: \[\hat\tau_{\mathrm{HJ}} = \frac{\sum_i a_i Y_i}{\sum_i a_i} - \frac{\sum_i c_i Y_i}{\sum_i c_i} = \frac{\sum_i T_i Y_i / \hat\pi(X_i)}{\sum_i T_i / \hat\pi(X_i)} - \frac{\sum_i (1-T_i) Y_i / (1-\hat\pi(X_i))}{\sum_i (1-T_i) / (1-\hat\pi(X_i))}.\] The same two forms exist for the ATT: the treated arm needs no weight, and a control receives the odds \(o_i = \hat\pi_i/(1-\hat\pi_i)\), \[\hat\tau_{\mathrm{HT,ATT}} = \frac{1}{n_1}\sum_{i:\,T_i=1} Y_i - \frac{1}{n_1}\sum_{i:\,T_i=0} o_iY_i, \qquad \hat\tau_{\mathrm{HJ,ATT}} = \frac{1}{n_1}\sum_{i:\,T_i=1} Y_i - \frac{\sum_{i:\,T_i=0} o_iY_i}{\sum_{i:\,T_i=0} o_i}.\] Both pairs are consistent for their targets under positivity and moment conditions; in finite samples they can differ a great deal.
Why Hájek: one identity. Let \(w_T = n^{-1}\sum_i a_i\) and \(w_C = n^{-1}\sum_i c_i\), each arm’s inverse-weight total divided by \(n\). By Lemma 6.4 with \(h \equiv 1\), each has expectation one under the true score, but each fluctuates. Shifting every outcome by a constant \(c\) gives \[\hat\tau_{\mathrm{HT}}(Y + c) = \hat\tau_{\mathrm{HT}}(Y) + c\,(w_T - w_C), \qquad \hat\tau_{\mathrm{HJ}}(Y + c) = \hat\tau_{\mathrm{HJ}}(Y). \tag{6.4}\] HT inherits the fluctuation of \(w_T - w_C\), multiplied by the level of the outcome; Hájek is location invariant because each arm’s self-normalized mean shifts by exactly \(c\). A unit with \(\pi(X_i) = 0.05\) has weight \(20\); with an outcome near a baseline of \(8\), its contribution to the HT sum is of order \(160\), and which units are extreme varies across samples. Normalization makes the estimate invariant to the outcome origin and often improves stability, which is why the Hájek form is the recommended default; but a single unit can still carry most of an arm’s weight, so extreme weights remain a problem for both estimators. The same holds for the ATT: the odds weight \(0.95/0.05 = 19\) on a control with a high score is as explosive as the ATE weight.
6.5.4 Why an Estimated Score Can Be More Precise Than the True One
Replacing the true score by a maximum-likelihood estimate from a correctly specified model does not merely preserve the precision of IPW; it improves it. The argument has the nuisance-parameter structure of Chapter 5, with one twist in the sign.
Setting. The observations \((T_i, X_i, Y_i)\) are i.i.d.; the score follows a correctly specified smooth parametric model \(\pi(X;\gamma)\), logistic for instance, with true value \(\gamma_0\), fitted by regular maximum likelihood on the same sample with nonsingular information; and \(\E[Y(1)^2/\pi(X) + Y(0)^2/\{1-\pi(X)\}] < \infty\). Let \(S_i = \partial \log p(T_i \mid X_i;\gamma_0)/\partial\gamma\) be the likelihood score vector — not the propensity score; the two meanings of “score” are both in play here — and \(\mathcal{I} = \E(S S^\top)\) the information. Write \[\varphi_i(\gamma) = \frac{T_iY_i}{\pi(X_i;\gamma)} - \frac{(1-T_i)Y_i}{1-\pi(X_i;\gamma)} - \tau_{\mathrm{ATE}}, \qquad \hat\tau_{\mathrm{HT}}(\hat\gamma) - \tau_{\mathrm{ATE}} = \frac{1}{n}\sum_{i=1}^n \varphi_i(\hat\gamma).\]
Expansion in the nuisance. Linearizing in \(\gamma\), \[\sqrt n\,\{\hat\tau_{\mathrm{HT}}(\hat\gamma) - \tau_{\mathrm{ATE}}\} = \underbrace{\frac{1}{\sqrt n}\sum_{i=1}^n \varphi_i(\gamma_0)}_{\text{known-score term}} + \underbrace{\Gamma\,\sqrt n\,(\hat\gamma - \gamma_0)}_{\text{estimation error}} + o_p(1), \qquad \Gamma = \E\{\partial\varphi(\gamma_0)/\partial\gamma^\top\},\] and the maximum-likelihood expansion gives \(\sqrt n(\hat\gamma - \gamma_0) = \mathcal{I}^{-1} n^{-1/2}\sum_i S_i + o_p(1)\). In Chapter 5 the estimation error of the outcome-regression coefficient was uncorrelated with the sampling error and added variance. Here the two terms are negatively related: the estimation error subtracts a piece of the known-score term.
Theorem 6.5 (Variance Reduction from an Estimated Score) Under the setting above, \(\Gamma = -\E(\varphi S^\top)\), the influence function of \(\hat\tau_{\mathrm{HT}}(\hat\gamma)\) is \[\psi = \varphi - \E(\varphi S^\top)\,\mathcal{I}^{-1} S = \varphi - \Pi(\varphi \mid S),\] \(\varphi\) minus its linear projection on the likelihood score vector, and the limiting variances of the \(\sqrt n\)-scaled estimators satisfy \[V_{\mathrm{est}} = \mathrm{Var}(\varphi) - \E(\varphi S^\top)\,\mathcal{I}^{-1}\,\E(S\varphi) \;\le\; V_{\mathrm{known}} = \mathrm{Var}(\varphi), \qquad \mathrm{Var}\{\hat\tau_{\mathrm{HT}}(\hat\gamma)\} \approx V_{\mathrm{est}}/n.\]
For the logistic model with regressors \(f(X)\), \(S_i = \{T_i - \pi(X_i)\}f(X_i)\), and the likelihood equations \(\sum_i\{T_i - \hat\pi(X_i)\}f(X_i) = 0\) hold exactly in the sample. They act as control variates: the part of the IPW error that is correlated with \(\{T - \pi(X)\}f(X)\) is removed, and the proportional variance reduction \(1 - V_{\mathrm{est}}/V_{\mathrm{known}}\) is the squared multiple correlation of \(\varphi\) with the components of \(S\). The remark on the saturated score is the extreme case, in which the removal is exact in every sample; in the lab below, the estimated score cuts the standard deviation of HT-ATE from \(0.626\) to \(0.194\). The gain requires a correctly specified score fitted by maximum likelihood (or a suitable nonparametric fit; the semiparametric efficiency result is Hirano et al. (2003)); it does not hold for an arbitrary estimate of \(\pi\), and it says nothing about misspecification, which removes the consistency guarantee altogether (Part 2 of the lab).
Variance estimation. With \(\hat\varphi_i = \varphi_i(\hat\gamma)\) evaluated at \(\hat\tau_{\mathrm{HT}}\), \(\hat S_i\) the fitted likelihood score vector, \(\hat{\mathcal I} = n^{-1}\sum_i \hat S_i\hat S_i^\top\), and \(\hat\Gamma = n^{-1}\sum_i \hat\varphi_i\hat S_i^\top\), the influence-function plug-in is \[\hat\psi_i = \hat\varphi_i - \hat\Gamma\,\hat{\mathcal I}^{-1}\hat S_i, \qquad \widehat{\mathrm{Var}}(\hat\tau_{\mathrm{HT}}) = \frac{1}{n^2}\sum_{i=1}^n \hat\psi_i^2,\] an estimate of \(V_{\mathrm{est}}/n\) under the stated model; using \(\hat\varphi_i\) alone, as if the score were known, is asymptotically conservative. For the Hájek estimator, a ratio in each arm, linearize as in Chapter 5 by centering each arm at its Hájek mean, \(\hat\varphi_{\mathrm{HJ},i} = T_i(Y_i - \hat\mu_1)/\hat\pi_i - (1-T_i)(Y_i - \hat\mu_0)/(1-\hat\pi_i)\) with no further \(-\hat\tau\), and project on \(\hat S_i\) as before. These are large-sample statements, not finite-sample guarantees. The bootstrap is also available for these regular parametric estimators: resample units and refit \(\hat\pi\) in every replication, since treating \(\hat\pi\) as fixed overstates the variance. With flexible or adaptive fitting methods, bootstrap validity needs separate justification, and for matching estimators the ordinary bootstrap fails.
6.5.5 Advanced Topic: Calibrated Estimation of the Propensity Score
Maximum likelihood is not the only way to fit the score. The calibration approach of Tan (2020) fits the same logistic model so that the inverse-probability weights balance the covariates exactly in the sample. Tan’s loss is easiest to understand as the dual of a calibration problem posed by Wang and Kim (2025). The construction is the survey calibration of Chapter 5 (Deville and Särndal 1992) with the treated arm as the sample and the whole data set as the population. With the control arm as the sample and the treated arm as the population, the same construction gives the ATT calibration known as entropy balancing, later in the section.
The primal problem. Let \(S = \{i : T_i = 1\}\) and \(f_i = f(X_i)\) with \(f(X) = (1, X^\top)^\top\). Choose weights \(\{\omega_i\}_{i \in S}\) and estimate \(\E\{Y(1)\}\) by \(n^{-1}\sum_{i \in S}\omega_i Y_i\). Wang and Kim (2025) choose the weights to solve \[\min_{\omega}\ \sum_{i\in S} (\omega_i - 1)\{\log(\omega_i - 1) - 1\} \qquad\text{subject to}\qquad \sum_{i\in S}\omega_i f_i = \sum_{i=1}^n f_i \tag{6.5}\] over \(\omega_i > 1\). The constraint is exact covariate balance: the reweighted treated arm reproduces the sample totals of \(f(X)\), the sample version of the first identity in Lemma 6.4. The objective is a shifted Kullback–Leibler entropy, and the shift has a direct reading. With \(r_i = \omega_i - 1\), the problem is to minimize \(\sum_{i\in S}(r_i\log r_i - r_i)\) subject to \(\sum_{i\in S} r_i f_i = \sum_{i:\,T_i=0} f_i\): each treated unit keeps a weight of one for itself, and the excess weights \(r_i\), as close to one as possible in Kullback–Leibler divergence, make the treated arm reproduce the control totals as well.
Write \(G(\omega) = (\omega-1)\{\log(\omega-1)-1\}\), so that \(G'(\omega) = \log(\omega - 1)\), and introduce a multiplier \(\lambda\) for the constraint. Differentiating the Lagrangian \(L(\omega,\lambda) = \sum_{i\in S}G(\omega_i) - \lambda^\top(\sum_{i\in S}\omega_i f_i - \sum_{i=1}^n f_i)\) with respect to \(\omega_i\) gives \(\log(\omega_i - 1) = \lambda^\top f_i\), so \[\hat\omega_i(\lambda) = 1 + \exp(\lambda^\top f_i) = \frac{1}{\expit(-\lambda^\top f_i)} = \frac{1}{\pi(X_i;\gamma)}, \qquad \gamma = -\lambda,\] the inverse of a logistic propensity score: the model is implied by the objective, not assumed.
The dual problem is Tan’s calibration loss. Substituting \(\hat\omega_i(\lambda)\) back into \(L\) leaves a problem in the low-dimensional \(\lambda\) alone. With \(u_i = \lambda^\top f_i\) and \(\hat\omega_i - 1 = \exp(u_i)\), \[G(\hat\omega_i) - u_i\,\hat\omega_i = \exp(u_i)(u_i - 1) - u_i\{1 + \exp(u_i)\} = -u_i - \exp(u_i),\] so \(L\{\hat\omega(\lambda),\lambda\} = -F(\lambda)\) with \[F(\lambda) = \sum_{i\in S}\{\lambda^\top f_i + \exp(\lambda^\top f_i)\} - \sum_{i=1}^n \lambda^\top f_i = \sum_{i=1}^n \Bigl[T_i \exp\{-\gamma^\top f_i\} + (1 - T_i)\,\gamma^\top f_i\Bigr] = \ell_{\mathrm{CAL}}(\gamma), \tag{6.6}\] the calibration loss of Tan (2020). \(F\) is convex, and \(\hat\lambda\) minimizes it. Its gradient vanishes if and only if \(\sum_i T_i f_i/\hat\pi_i = \sum_i f_i\), the primal constraint: exact balance is not a by-product of this fit, it is what the weights were chosen to satisfy. Because \(f\) contains the constant, \(\sum_i T_i/\hat\pi_i = n\), so the Horvitz–Thompson and Hájek estimators coincide. Logistic maximum likelihood minimizes \(-\sum_i\{T_i\log\pi_i + (1-T_i)\log(1-\pi_i)\}\) instead; its score equation \(\sum_i(T_i - \hat\pi_i)f_i = 0\) balances only on the probability scale. The control arm is handled by the mirror-image loss \(\sum_i[(1-T_i)\exp\{\gamma^\top f_i\} - T_i\,\gamma^\top f_i]\), whose first-order condition is \(\sum_i(1-T_i)f_i/(1-\hat\pi_i) = \sum_i f_i\); it returns below as the dual of the ATT calibration.
Proposition 6.1 (What Exact Balance Buys) Suppose \(\E\{Y(1) \mid X\} = \beta^\top f(X)\) for some \(\beta\), and let \(\hat\omega_i\) be weights that satisfy the constraint of Equation 6.5 — the calibrated weights in particular — with \(\hat\omega_i \to \omega^*(X_i)\) uniformly for some function \(\omega^*\). Under consistency, \(Y(1) \indep T \mid X\), and moment conditions, \(n^{-1}\sum_{i\in S}\hat\omega_i Y_i \pto \E\{Y(1)\}\) whether or not the logistic score model implied by the weights is correct.
Together with the usual IPW argument — if the implied score model is correct, the weights converge to the true inverse probabilities and consistency follows from Theorem 6.4 — this is the double robustness of Chapter 11, obtained here at the fitting stage of the score rather than by augmenting the estimator. The balanced weights annihilate the linear part of the outcome, and the residual term has mean zero under unconfoundedness whatever \(\hat\pi\) converges to: the Chapter 5 argument for regression weights once more.
Calibration for the ATT. For the ATT only the control mean \(\E\{Y(0) \mid T{=}1\}\) needs weighting, and the population identity to reproduce in the sample is the third one in Lemma 6.4, in its unnormalized form \(\E\{(1-T)\,o(X)\,h(X)\} = \E\{T\,h(X)\}\) with \(o(X) = \pi(X)/\{1-\pi(X)\}\). Applied to \(h = f\), it calibrates the control arm to the treated totals. Let \(S_0 = \{i : T_i = 0\}\), and estimate \(\E\{Y(0) \mid T{=}1\}\) by \(n_1^{-1}\sum_{i\in S_0}\omega_i Y_i\), with weights solving \[\min_{\omega}\ \sum_{i\in S_0} (\omega_i\log\omega_i - \omega_i) \qquad\text{subject to}\qquad \sum_{i\in S_0}\omega_i f_i = \sum_{i:\,T_i=1} f_i \tag{6.7}\] over \(\omega_i > 0\). The objective is the Kullback–Leibler divergence of the weights from the unit weight, up to an additive constant, now without the shift: the ideal ATT weights are the odds, with no unit of weight reserved for the control itself. Stationarity of the Lagrangian gives \(\log\omega_i = \lambda^\top f_i\), so \(\hat\omega_i = \exp(\lambda^\top f_i) = \pi(X_i;\gamma)/\{1 - \pi(X_i;\gamma)\}\) with \(\gamma = \lambda\), the odds under a logistic score. With \(G(\omega) = \omega\log\omega - \omega\), \(G(\hat\omega_i) - \lambda^\top f_i\,\hat\omega_i = -\exp(\lambda^\top f_i)\), and the dual objective is \[F(\lambda) = \sum_{i\in S_0}\exp(\lambda^\top f_i) - \sum_{i:\,T_i=1}\lambda^\top f_i = \sum_{i=1}^n \Bigl[(1 - T_i)\exp\{\gamma^\top f_i\} - T_i\,\gamma^\top f_i\Bigr], \tag{6.8}\] which is exactly the control-arm loss of the ATE. Its gradient vanishes if and only if the constraint of Equation 6.7 holds. Because \(f\) contains the constant, \(\sum_{i\in S_0}\hat\omega_i = n_1\), so the Horvitz–Thompson and Hájek forms of the ATT estimator coincide, as they did for the ATE.
The coincidence of the two losses is the identity \(1/(1-\pi) = 1 + \pi/(1-\pi)\). The ATE control-arm constraint \(\sum_{i\in S_0}\{1 + \exp(\gamma^\top f_i)\}f_i = \sum_{i=1}^n f_i\) becomes, after \(\sum_{i\in S_0} f_i\) is subtracted from both sides, the ATT constraint \(\sum_{i\in S_0}\exp(\gamma^\top f_i)f_i = \sum_{i:\,T_i=1} f_i\). The two fits therefore share \(\hat\gamma\), and on the controls \(\hat\omega_i^{\mathrm{ATE}} = 1 + \hat\omega_i^{\mathrm{ATT}}\): one fit calibrates both weighting schemes. The shift by one in the objective of Equation 6.5 is this identity.
Entropy balancing. Hainmueller (2012) proposed exactly Equation 6.7, up to the normalization of the weights: choose the control weights closest to uniform in Kullback–Leibler divergence subject to reproducing the treated moments of \(f(X)\). The logistic form of the odds is implied by the objective, not assumed, and Zhao and Percival (2017) identify the dual Equation 6.8 and prove the double robustness stated next. Two practical points follow from the form of the weights. They are positive by construction, unlike the regression weights of Chapter 5, which can be negative. And positive weights that satisfy the constraint exist only if the treated mean of the covariates lies inside the convex hull of the control covariate values; when it does not, the dual Equation 6.8 is unbounded below and has no minimizer. This is the sample-level face of an ATT overlap failure, \(\pi(x)\) near one where the treated units are (Section 6.6.4).
What exact balance buys for the ATT. The analogue of Proposition 6.1 holds with one-sided assumptions. Suppose \(\E\{Y(0) \mid X\} = \beta^\top f(X)\), let \(\hat\omega_i\) be weights that satisfy the constraint of Equation 6.7, with a uniform limit \(\omega^*(X_i)\), and assume consistency, \(Y(0) \indep T \mid X\), \(P(T{=}1) > 0\), and moment conditions. Write \(\varepsilon_i = Y_i(0) - \beta^\top f_i\), so that \(Y_i = \beta^\top f_i + \varepsilon_i\) for the controls. By the constraint, \[\frac{1}{n_1}\sum_{i\in S_0}\hat\omega_i Y_i = \frac{1}{n_1}\sum_{i:\,T_i=1}\beta^\top f_i + \frac{n}{n_1}\cdot\frac{1}{n}\sum_{i=1}^n (1-T_i)\,\hat\omega_i\, \varepsilon_i .\] The first term converges to \(\E\{m_0(X) \mid T{=}1\} = \E\{Y(0) \mid T{=}1\}\), by ignorability as in the proof of Corollary 6.1. In the second, \(n/n_1 \to 1/P(T{=}1)\), and \(\E\{(1-T)\,\omega^*(X)\,\varepsilon\} = \E[\omega^*(X)\{1-\pi(X)\} \E(\varepsilon \mid X, T{=}0)] = 0\), because \(\E(\varepsilon \mid X, T{=}0) = \E(\varepsilon \mid X) = 0\) under \(Y(0) \indep T \mid X\). Hence the calibrated estimator of \(\E\{Y(0) \mid T{=}1\}\) is consistent if the outcome model for \(Y(0)\) is linear in \(f\); if instead the logistic score is correct, \(\hat\omega_i \to o(X_i)\) and Corollary 6.1 applies. Only the control-arm outcome model and \(Y(0) \indep T \mid X\) enter, matching the one-sided assumptions of the ATT.
Example 6.7 (The Calibration Loss on the Lab’s Misspecified Design) Part 2 of the lab (Section 6.7.2) fits a linear logistic model to a treatment mechanism that is quadratic in \(X_1\), while the outcome is linear in \(X\). Fitting the same linear logit by the calibration loss instead of maximum likelihood, in a separate run of \(1{,}000\) replications at \(n = 1{,}000\) (script cal18.R, seed 18), gives the following for the ATE, whose true value is \(1\), and for the ATT, whose true value is \(1.152\). The ATT row of the calibration uses the control-arm fit of the ATE, that is, entropy balancing:
| Fit of the linear logit | Estimand | Mean | SD |
|---|---|---|---|
| Maximum likelihood, HT | ATE | 1.477 | 0.153 |
| Maximum likelihood, Hájek | ATE | 0.862 | 0.088 |
| Calibration loss (HT \(=\) Hájek) | ATE | 1.002 | 0.070 |
| Maximum likelihood, HT | ATT | 1.773 | 0.166 |
| Maximum likelihood, Hájek | ATT | 1.304 | 0.083 |
| Entropy balancing (HT \(=\) Hájek) | ATT | 1.152 | 0.079 |
The maximum-likelihood rows reproduce the biases of Table 6.3. The calibrated estimates are essentially unbiased for both estimands — not because the score model has become right, but because the outcome is linear in \(f(X)\) and the weights balance \(f(X)\) exactly (to \(10^{-7}\) in every replication), which is the situation of Proposition 6.1 and of its ATT analogue.
6.6 Overlap and Positivity
6.6.1 Positivity and Strong Overlap
We must distinguish two conditions on the propensity score that play different roles in identification and inference.
Definition 6.4 (Positivity (Weak Overlap)) The positivity condition — also called weak overlap — requires \[0 < \pi(X) < 1 \quad \text{almost surely.}\] Every unit has a positive probability of receiving either treatment or control — more precisely, this holds for \(P_X\)-almost every covariate value.
Together with unconfoundedness, positivity constitutes the strong ignorability of Rosenbaum and Rubin (1983). Positivity is the condition needed for identification: without it, certain strata have no counterfactual information and the ATE is not nonparametrically identified.
Definition 6.5 (Strong Overlap) The strong overlap condition requires that there exists a positive constant \(c \in (0, 1/2)\) such that \[c \le \pi(X) \le 1 - c \quad \text{almost surely.}\]
Strong overlap bounds the inverse weights \(1/\pi(X)\) and \(1/(1-\pi(X))\) uniformly away from infinity. Bounded weights do not bound the outcome, so strong overlap is only part of what \(\sqrt{n}\) inference requires. For the known-score Horvitz–Thompson term, a useful sufficient moment condition is \[\E\!\left[\frac{Y(1)^2}{\pi(X)} + \frac{Y(0)^2}{1 - \pi(X)}\right] < \infty, \tag{6.9}\] which strong overlap together with \(\E\{Y(t)^2\} < \infty\) guarantees. Together with finite outcome second moments and the regularity conditions of the estimator in use, strong overlap therefore supports the finite-variance, \(\sqrt{n}\)-consistent, asymptotically normal theory of later chapters (Chapter 12). It is sufficient for controlling the denominators, but it is not necessary: root-\(n\) inference can survive under weaker, estimator-specific tail and moment conditions, provided the relevant influence function retains finite variance. Positivity alone permits identification but does not guarantee stable inference: when \(\pi(X)\) approaches \(0\) or \(1\) in parts of the covariate space, the moment Equation 6.9 can be infinite, the IPW weights blow up, a few units dominate, and the \(\sqrt{n}\) rate fails even though the parameter is identified (Khan and Tamer 2010). Part 2 of the lab contains an example in which positivity holds, strong overlap fails, and the known-score HT estimator has infinite variance.
Why positivity is necessary. The identification formula \[\tau_{\mathrm{ATE}} = \E_X\bigl[\E[Y \mid T{=}1, X] - \E[Y \mid T{=}0, X]\bigr]\] requires that both \(\E[Y \mid T{=}1, X{=}x]\) and \(\E[Y \mid T{=}0, X{=}x]\) be defined for \(P_X\)-almost all \(x\) in the support of \(X\). If \(\pi(x) = 0\) on a set of covariate values of positive probability, then no treated units exist there, so \(\E[Y \mid T{=}1, X{=}x]\) is undefined on that set and the ATE is not identified by adjustment without additional assumptions about \(Y(1)\); similarly if \(\pi(x) = 1\). Three grades should be kept apart: a violation on a \(P_X\)-null set is harmless; a violation on a set of positive mass is an identification failure; and a score that is small but strictly positive is not an identification failure at all, but an estimation problem — limited information, extreme weights, and possibly infinite variance — which is the subject of the rest of this section.
6.6.2 Practical Consequences of Near-Violations
Exact violations of overlap (\(\pi(X) \in \{0,1\}\) on a set of positive mass) arise from eligibility criteria and deterministic treatment rules. Near-violations arise when one treatment is uncommon in parts of covariate space: some covariate strata have very few treated or control units, making the estimated propensity score close to \(0\) or \(1\).
Near-violations arise when a covariate is a near-perfect predictor of treatment, when the covariate distributions of treated and control groups barely overlap, or when the propensity score model is over-flexible and fits the treatment variable nearly perfectly in-sample. Severe lack of overlap is therefore not necessarily a model-fitting failure: it may reflect a genuine absence of comparable treated and control units in the target population, in which case the honest responses are those of the next two subsections — trimming or re-targeting the estimand — rather than model revision (cf. step 7 of the workflow in Section 6.3.7).
6.6.3 Trimming Strategies
Trimming by propensity score. Restrict the analysis to units with estimated propensity scores in a prespecified interval \([\eta, 1-\eta]\) for some small \(\eta > 0\). The estimand changes to the ATE in the trimmed subpopulation: \[\tau_{\mathrm{trim}} = \E\bigl[Y(1) - Y(0) \mid \eta \le \pi(X) \le 1-\eta\bigr].\] Note that the target population is defined by the true score \(\pi(X)\), while the practical rule is applied with the estimated score \(\hat\pi(X)\). With a consistent score estimate and negligible probability mass at the cutoffs, the two rules agree asymptotically; this does not make the uncertainty from estimating the rule negligible for inference, which must account for it (Yang and Ding 2018). The choice of \(\eta\) is a precision–target-population trade-off rather than a bias–variance trade-off in the usual sense: removing regions with extreme scores eliminates the largest weights, but it also discards observations and changes the estimand. The discrepancy between \(\tau_{\mathrm{trim}}\) and the original ATE is a change of target, not a statistical bias relative to \(\tau_{\mathrm{trim}}\), and the net variance need not decrease monotonically in \(\eta\).
The Crump et al. rule. Crump et al. (2009) derive the subpopulation that minimizes the asymptotic variance of the ATE estimator under their criterion and assumptions; the simple symmetric interval \([\eta, 1-\eta]\) uses additional conditions, homoskedasticity among them. Cutoffs near \(0.1\) often emerge from this calculation, so the fixed rule \(\eta = 0.1\) — trimming units with \(\hat\pi(X) < 0.1\) or \(\hat\pi(X) > 0.9\) — is a common heuristic, though not an optimum for every estimator and outcome model.
6.6.4 Re-targeting the Estimand to Avoid Overlap Failures
Trimming restricts the population in order to preserve the ATE within a region of good overlap. An alternative, complementary response to overlap failure is to change which units the effect averages over: target the average treatment effect on a subset of the population (treated or control) for which the relevant side of positivity can be guaranteed.
ATT (target the treated population). The ATT averages \(Y(1) - Y(0)\) over \(X \mid T{=}1\) and therefore requires only that \(\E[Y(0) \mid X = x]\) be identified where the treated population lives. The treated-covariate law is absolutely continuous with respect to \(P_X\) with density proportional to the propensity score, \(dP_{X \mid T=1} = \{\pi(x)/P(T{=}1)\}\, dP_X\), so treated-population mass concentrates on \(\{x : \pi(x) > 0\}\); there the requirement reduces to \(P(T{=}0 \mid X{=}x) > 0\), equivalently \(\pi(x) < 1\). Hence, given consistency, \(Y(0) \indep T \mid X\), and integrability, the remaining support condition for nonparametric identification by adjustment is that \(\pi(X) < 1\) almost surely on the support of \(X \mid T{=}1\). The ATT therefore tolerates regions with \(\pi(x) = 0\) (no treated units are observed there in any case, so \(X = x\) contributes nothing to the treated-population average); the ATT cannot tolerate regions inside the treated support where \(\pi(x) = 1\) (no comparable controls).
ATC (target the control population). Symmetrically, given consistency, \(Y(1) \indep T \mid X\), and integrability, the ATC averages \(Y(1) - Y(0)\) over \(X \mid T{=}0\) and is nonparametrically identified by adjustment if and only if \(\pi(X) > 0\) almost surely on the support of \(X \mid T{=}0\). The ATC tolerates regions where \(\pi(x) = 1\) but not regions inside the control support where \(\pi(x) = 0\). These equivalences are statements about the support conditions for nonparametric identification by adjustment, once consistency and the relevant arm-specific ignorability condition are in place; positivity by itself is never a complete identification condition. Additional outcome-model restrictions (e.g., a parametric model extrapolating into an unsupported region) could identify the same estimands under weaker support conditions, but then the model, not the data, supplies the answer.
Practical rule, conditional on the question. When overlap fails at covariate values where \(\pi(X) \approx 0\) — treated units are scarce or absent there — and the treated population is the scientifically relevant one, the ATT is a more feasible target. When overlap fails where \(\pi(X) \approx 1\) — controls are scarce or absent — the analogous consideration points to the ATC. When overlap fails on both sides, restrict to the trimmed-population estimand of the previous subsection, or use the overlap weights of the remark in Section 6.3.6. Re-targeting should answer a scientific question, not be an automatic response to whichever estimate is more stable; every remedy for an overlap failure changes the question, and the analysis must say which question is answered.
6.7 Lab: Simulation Study of IPW and Matching Estimators
This lab compares five estimators on a five-covariate data-generating process with treatment-effect heterogeneity. With five continuous confounders, direct stratification is infeasible, making propensity-score methods the natural choice. The estimators span two estimands: the ATE (targeted by the HT and Hájek IPW estimators) and the ATT (targeted by the ATT-IPW variants and by matching). Five questions drive the comparison:
- How much variance does weight normalization save (Hájek over HT) for ATE estimation, and why does the baseline outcome mean matter?
- Does the same variance problem afflict HT when targeting the ATT, and does Hájek normalization help as much?
- Among estimators targeting the ATT, do IPW and matching agree on the target, and which is more efficient?
- How large is the estimand gap between ATT and ATE, and which estimators reveal it?
- How does replacing the true propensity score with a logistic estimate change bias and variance?
6.7.1 Part 1: Correctly Specified Propensity Score
With five confounders, direct cell-by-cell stratification is infeasible: five continuous covariates cannot be coarsened without discarding information or requiring enormous samples. The propensity score reduces this five-dimensional adjustment to a scalar. The large baseline mean (\(\E[Y(0)] = 8\)) plays a distinct role: it amplifies the consequences of extreme IPW weights through Equation 6.4, making Hájek normalization strongly recommended in this setting rather than merely helpful, as the results below demonstrate.
Estimators. Five estimators are compared, each applied under both the true and estimated propensity score. The estimated score \(\hat\pi(X)\) is obtained by logistic regression of \(T\) on \((X_1, \ldots, X_5)\), which is correctly specified since the true logit is linear in all five covariates.
Horvitz–Thompson ATE (HT-ATE) and Hájek ATE (HJ-ATE) are the two forms of Section 6.5.3, targeting the ATE. Raw inverse weights can be extreme when \(\hat\pi(X_i)\) is near \(0\) or \(1\), and with \(\E[Y(0)]=8\) the variance consequences are severe; normalizing by the empirical weight sum within each arm makes the estimate invariant to the outcome origin.
Horvitz–Thompson ATT (HT-ATT) and Hájek ATT (HJ-ATT) re-weight the control arm to look like the treated population while leaving the treated arm unchanged. Each control unit receives weight proportional to its odds of treatment \(\hat\pi(X_i)/(1-\hat\pi(X_i))\): units whose covariates resemble the treated population receive large weight. For a control unit with \(\hat\pi(X_i) = 0.95\) the weight is \(19\); combined with a baseline outcome near \(8\), this produces extreme variance — in practice, even worse than HT-ATE. Because it divides by the random \(n_1\), HT-ATT is not exactly unbiased in finite samples. The treated arm is already a simple mean and requires no normalization; the Hájek adjustment applies only to the control arm. HJ-ATT is the recommended default estimator for the ATT in practice.
One-to-one nearest-neighbor matching with replacement (NNM) pairs each treated unit with the control unit having the closest propensity score, with reuse allowed (Abadie and Imbens 2006), and targets the ATT. Matching with replacement ensures every treated unit finds its closest control, avoiding the finite-sample “running out of good matches” problem that arises without replacement when the propensity score distribution is wide; by Equation 6.2 it reweights the controls by their reuse counts.
Results. Table 6.2 reports the Monte Carlo mean, bias, standard deviation, and RMSE across \(B = 2{,}000\) replications at \(n = 1{,}000\), seed 42. Bias is computed relative to each estimator’s own target: ATE \(= 1.000\) for HT-ATE and HJ-ATE; ATT \(\approx 1.199\) for HT-ATT, HJ-ATT, and NNM; entries are rounded from the simulation output, so a bias may differ in the last digit from the difference of the rounded mean and target. Monte Carlo standard errors of the reported means equal \(\mathrm{SD}/\sqrt{B} \approx \mathrm{SD}/45\), here and in Part 2; reported biases should be read against this yardstick.
42.
| PS | Estimator | Estimand | Mean | Bias | SD | RMSE |
|---|---|---|---|---|---|---|
| Known | HT-ATE | ATE | 1.008 | $+$0.008 | 0.626 | 0.626 |
| HJ-ATE | ATE | 1.003 | $+$0.003 | 0.145 | 0.145 | |
| HT-ATT | ATT | 1.184 | $-$0.016 | 0.725 | 0.725 | |
| HJ-ATT | ATT | 1.203 | $+$0.003 | 0.131 | 0.131 | |
| NNM | ATT | 1.207 | $+$0.008 | 0.130 | 0.130 | |
| Estimated | HT-ATE | ATE | 0.998 | $-$0.002 | 0.194 | 0.194 |
| HJ-ATE | ATE | 1.002 | $+$0.002 | 0.102 | 0.103 | |
| HT-ATT | ATT | 1.202 | $+$0.002 | 0.251 | 0.251 | |
| HJ-ATT | ATT | 1.202 | $+$0.002 | 0.101 | 0.101 | |
| NNM | ATT | 1.205 | $+$0.006 | 0.121 | 0.121 |
Five lessons stand out.
1. Hájek normalization is strongly recommended for ATE estimation when outcomes are non-centered. With the known propensity score, HT-ATE achieves SD \(= 0.626\), while HJ-ATE achieves SD \(= 0.145\) — a 4.3-fold reduction in SD from simply normalizing the weights. The mechanism is Equation 6.4: a unit with \(\pi(X_i) = 0.05\) receives raw weight \(1/0.05 = 20\); multiplied by an outcome near \(\E[Y(0)] = 8\), its contribution to the HT sum is of order \(160\). Variability in which units are extreme across samples enters HT through the fluctuation of \(w_T - w_C\), amplified eightfold by the baseline, and inflates the HT standard deviation to more than half the true effect size; this, not variability in the treatment effect itself, is the dominant source of the HT variance here. The Hájek difference is exactly invariant to the shift, since each arm’s self-normalized mean moves by \(c\) and the shifts cancel. As the remark on normalization warns, this is a property of non-centered outcomes with fluctuating weight sums, not a universal variance ordering.
2. HT-ATT is even more unstable; Hájek normalization is equally beneficial for ATT estimation. With the known PS, HT-ATT achieves SD \(= 0.725\), slightly worse than HT-ATE (SD \(= 0.626\)). The reason is structural: the ATT re-weights the control arm by the treatment odds \(\pi(X)/(1-\pi(X))\). A control unit with \(\pi(X) = 0.95\) receives an odds-ratio weight of \(0.95/0.05 = 19\); combined with the large baseline outcome this is as explosive as the ATE inverse-probability weight. The apparent bias of \(-0.016\) for HT-ATT is within one Monte Carlo standard error of \(0.725/\sqrt{2000} \approx 0.016\) and should be attributed to simulation noise together with the finite-sample bias of dividing by the random \(n_1\), not to inconsistency. HJ-ATT reduces SD to \(0.131\) — a 5.5-fold reduction in SD — confirming that Hájek normalization is not an ATE-specific fix but a general principle applicable whenever inverse-probability-style weights are used.
3. HJ-ATT and NNM converge to the same target with nearly identical efficiency under the known score. Under the known PS, HJ-ATT and NNM both target the ATT \(\approx 1.199\) and achieve essentially the same standard deviation (0.131 vs. 0.130). This is the sharpest demonstration of the “two routes, one estimand” principle: IPW re-weights the full sample using the propensity score; matching discards some control units and retains only those nearest each treated unit on the propensity scale. Both are consistent for the ATT under strong ignorability, and at \(n = 1{,}000\) they are virtually indistinguishable in efficiency. The practical implication is that the choice between IPW-ATT and matching can be made on secondary grounds — interpretability, balance diagnostics, or computational cost — rather than efficiency when the PS is known.
4. The estimand gap between ATE and ATT is large and clearly revealed by the table. The ATE estimators (HT-ATE, HJ-ATE) converge to \(1.000\); the ATT estimators (HT-ATT, HJ-ATT, NNM) converge to \(\approx 1.199\). The \(0.199\) gap is not a bias — it is a real difference in the estimands. High-\(X_1\) units are more likely to be treated (Equation 6.10) and have larger treatment effects (Equation 6.11); the treated subpopulation therefore benefits more than the full population average. A researcher who applies NNM and reports its output against the ATE benchmark of \(1.000\) would conclude the estimator has \(20\%\) bias; in fact it is unbiased for the correct target. Declaring the estimand before looking at any results is a non-negotiable discipline.
5. The estimated PS collapses the HT variance and makes HJ-ATT the most precise ATT estimator in this design. Under the estimated PS, the HT-ATE SD falls from \(0.626\) to \(0.194\) and the HT-ATT SD from \(0.725\) to \(0.251\): reductions of roughly two thirds in standard deviation, corresponding to variance reductions of about \(90\%\) and \(88\%\). This counterintuitive gain — the estimated score outperforming the true one — is Theorem 6.5: the logistic model is exactly correct here and is fitted by maximum likelihood, so the likelihood equations \(\sum_i \bigl(T_i - \hat\pi(X_i)\bigr)X_i = 0\) act as control variates and remove the component of the IPW sampling error correlated with the likelihood score vector. The Hájek-versus-HT ratios of Lessons 1 and 2 are correspondingly smaller with the estimated score (about \(1.9\) and \(2.5\)), because part of what normalization removed is now removed by the fit. More strikingly, HJ-ATT with the estimated PS achieves SD \(= 0.101\) — below both NNM (SD \(= 0.121\)) and HT-ATT (SD \(= 0.251\)) and matching HJ-ATE (SD \(= 0.102\)) almost exactly. A properly normalized IPW estimator for the ATT is consistent — though, as a ratio estimator, not exactly unbiased in finite samples — and can be more precise than matching when the propensity score model is correct and well-estimated. This ranking is specific to the data-generating process and implementations compared: correct specification is not semiparametric efficiency, and there is no universal variance ordering between the two estimators.
6.7.2 Part 2: Robustness to PS Model Misspecification
Part 1 established that under a correctly specified PS model, IPW and NNM are both consistent for their respective estimands, with HJ-ATT and NNM delivering similar efficiency. Part 2 breaks that assumption by introducing a nonlinear true propensity score that the standard logistic regression cannot capture.
Modified DGP. The outcome model is unchanged from Equation 6.11. The true propensity score now contains a quadratic term in \(X_1\), and \(X_2\) and \(X_3\) do not affect treatment: \[\pi^*(X) = \expit\{\ell(X)\}, \qquad \ell(X) = -0.5 + 0.8X_1 + 0.5X_1^2 + 0.2X_4 + 0.1X_5 . \tag{6.14}\] The quadratic \(0.5X_1^2\) means that both large positive and large negative values of \(X_1\) increase the probability of treatment, creating a U-shaped propensity surface. Because \(\E[X_1] = 0\) and the CATE is still \(\tau(X) = 1 + 0.5X_1\), \(\tau_{\mathrm{ATE}} = 1.000\) exactly, unchanged. The ATT changes because the new PS shifts which units are treated: \[\tau_{\mathrm{ATT}}^* \approx 1.152 \quad (\text{Gaussian quadrature: } 1.15151). \tag{6.15}\] The estimated PS is still obtained by fitting a linear logistic regression of \(T\) on \((X_1, X_2, X_3, X_4, X_5)\). This model is misspecified: it omits the \(X_1^2\) term and includes \(X_2\) and \(X_3\), which do not appear in the true logit.
Results. Table 6.3 reports the same five estimators as Part 1, now applied under the modified DGP. The “True PS” rows use \(\pi^*(X)\) from Equation 6.14; the “Misspecified” rows use a linear logistic estimate \(\hat\pi(X)\) that omits \(X_1^2\). The probability limits of the misspecified-score IPW estimators can be computed by solving the population logistic score equations and integrating: \(1.474\) (HT-ATE), \(0.860\) (HJ-ATE), \(1.776\) (HT-ATT), and \(1.305\) (HJ-ATT), in agreement with the Monte Carlo means.
42. The true-PS SDs are seed-dependent finite-sample numbers (see the warning above).
| PS | Estimator | Estimand | Mean | Bias | SD | RMSE |
|---|---|---|---|---|---|---|
| True | HT-ATE | ATE | 1.028 | $+$0.028 | 0.824 | 0.825 |
| HJ-ATE | ATE | 1.013 | $+$0.013 | 0.152 | 0.152 | |
| HT-ATT | ATT | 1.186 | $+$0.034 | 1.333 | 1.333 | |
| HJ-ATT | ATT | 1.183 | $+$0.031 | 0.212 | 0.215 | |
| NNM | ATT | 1.178 | $+$0.026 | 0.175 | 0.177 | |
| Misspecified | HT-ATE | ATE | 1.478 | $+$0.478 | 0.147 | 0.501 |
| HJ-ATE | ATE | 0.861 | $-$0.139 | 0.087 | 0.164 | |
| HT-ATT | ATT | 1.780 | $+$0.628 | 0.162 | 0.649 | |
| HJ-ATT | ATT | 1.306 | $+$0.154 | 0.081 | 0.174 | |
| NNM | ATT | 1.172 | $+$0.020 | 0.138 | 0.139 |
Three lessons complete the picture.
6. A misspecified PS removes the consistency guarantee for IPW; normalization does not restore it. Every IPW estimator using the misspecified score in this simulation is inconsistent: HT-ATE bias is \(+0.478\), nearly half the true effect; HJ-ATE bias is \(-0.139\); HT-ATT bias is \(+0.628\); HJ-ATT bias is \(+0.154\). Comparing these with the near-zero biases in the “True PS” rows makes clear that Hájek normalization does not generally restore consistency: a structurally wrong model produces structurally wrong weights. IPW consistency is generally tied to consistency of the PS estimator; the exceptions, such as a constant shift of the logit in the Hájek ATT estimator (Section 6.4.4), are special. (The protection that does exist is specific and must be built in: Section 6.5.5 fits the same linear logit so that the weights balance \(X\) exactly, and because the outcome is linear in \(X\) the resulting estimate is unbiased, \(1.002\) with SD \(0.070\), in Example 6.7. Ordinary maximum likelihood provides no such protection here.)
Notice also that HJ-ATE and HT-ATE have biases in opposite directions (\(-0.139\) vs \(+0.478\)): under misspecification the two estimators converge to different quantities, so normalization changes the limit even though it does not repair it. The mechanism is a multiplicative level error that HT inherits from misspecification. Recall the average inverse weights \(w_T\) and \(w_C\) from Section 6.5.3, each of which converges to \(1\) under a correctly specified PS. Under misspecification both can drift; in this DGP their limits are \(w_T \approx 1.045\) and \(w_C \approx 0.972\). The exact finite-sample relation is \[\hat\tau_{\mathrm{HT}} - \hat\tau_{\mathrm{HJ}} = (w_T - 1)\,\hat\mu_1^{\mathrm{HJ}} - (w_C - 1)\,\hat\mu_0^{\mathrm{HJ}},\] which evaluates to about \(0.613\) in the limit; replacing the Hájek arm means by \(\E[Y(1)] = 9\) and \(\E[Y(0)] = 8\) gives the approximation \(0.045 \times 9 + 0.028 \times 8 \approx 0.63\). Either way this accounts for essentially the entire gap between the two biases (\(+0.478 - (-0.139) = +0.617\)). With \(\E[Y(0)] = 8\), even sub-five percent weight-sum errors translate into bias contributions of order \(0.3\). What remains in HJ-ATE is a smaller shape error: the residual misweighting of the population that survives self-normalization. The level and shape errors are determined by different aspects of the misspecification — the first by how far the average inverse weights drift from one, the second by how the misspecified weights re-weight the population — so there is no reason for their signs to align.
7. In this simulation, matching is far less biased than IPW under the misspecified score, and the prognostic score offers an explanation. The NNM bias barely changes: \(+0.026\) under the true PS versus \(+0.020\) under the misspecified PS, and its RMSE is \(0.139\). The linear logistic model, though wrong, retains the dominant linear dependence of treatment on \(X_1\), \(X_4\), and \(X_5\), and in this design the matched pairs it produces remain close on the covariates that enter the outcome. That is an observation about this simulation, not a guarantee: close values of an arbitrary fitted scalar do not imply close covariate vectors or conditional exchangeability given that scalar, and a finite list of small standardized differences does not prove that the relevant population conditions hold. Before attributing a small bias to successful adjustment, assess balance on the outcome-relevant covariates and their transformations in the matched sample. Section 6.4.4 offers an explanation: after the outcome-only covariates \(X_2\) and \(X_3\) are set aside, the misspecified linear index is nearly a prognostic score for the remaining adjustment set \((X_1, X_4, X_5)\). The explanation is not an exact guarantee.
What matching needs can be stated precisely. For matching on a limiting scalar \(b(X)\), a sufficient causal condition for the ATT is \[\E\{Y(0) \mid b(X),\, T{=}1\} = \E\{Y(0) \mid b(X),\, T{=}0\},\] together with the overlap, shrinking-discrepancy, moment, and regularity conditions of the warning on inference after matching. A correctly estimated propensity score is one way to obtain this condition, since the true score is a balancing score. By Lemma 6.3, a misspecified score also provides it when it keeps the level sets of the true score (a wrong link, for instance) or of a prognostic score, even though it is not a correctly calibrated treatment probability. It does not provide it when the misspecification merges units with different scores and different prognoses, and then matching fails along with IPW (Example 6.6). The lesson is that the identifying condition for matching survives a larger class of misspecifications than the consistency of IPW does, because it depends on which units the score treats as alike rather than on the score’s values. The finite-sample matches and any caliper still depend on the score’s scale (Section 6.4.4), and whether a given fit falls in that class is checked through balance after matching, not assumed. Related constructions in the literature should not be over-read: Yang and Zhang (2023) obtain a multiply robust matching estimator by matching on both propensity and prognostic scores (Section 6.4.3) with a specific construction, which is a property of that estimator rather than a robustness guarantee for ordinary matching on one misspecified propensity score; Yang et al. (2016) extend score matching and stratification to multi-level treatments.
8. In this simulation, the efficiency ranking of matching and IPW reverses with model misspecification. In Part 1 (correctly specified PS), HJ-ATT had a smaller empirical SD than NNM under the estimated PS (\(0.101\) vs \(0.121\)). In Part 2 (misspecified PS), NNM has the smaller empirical RMSE (\(0.139\) against \(0.174\) for HJ-ATT), and the smaller bias. The comparison is summarized in Table 6.4. These rankings are specific to the data-generating processes and implementations compared. The practical implication is that the efficiency advantage of IPW is contingent on the PS model being approximately correct. When there is reason to doubt the functional form of the propensity model — for instance, when treatment is driven by a continuous covariate in a nonlinear way — matching can be a more robust alternative, because it uses the fitted score only to decide which units are alike, not as a weight (Section 6.4.4); whether this advantage materializes depends on whether the fitted score keeps the level sets of a balancing or a prognostic score, which balance on the outcome-relevant covariates after matching helps to assess.
| Part 1: Correct PS model | Part 2: Misspecified PS model | |||
|---|---|---|---|---|
| Estimator | Bias | RMSE | Bias | RMSE |
| HJ-ATT | \(+0.002\) | \(0.101\) | \(+0.154\) | \(0.174\) |
| NNM | \(+0.006\) | \(0.121\) | \(+0.020\) | \(0.139\) |
6.8 The Limits of Propensity Score Methods
6.8.1 The Untestable Assumption
The balancing property \(T \indep X \mid \pi(X)\) (Theorem 6.1), the coarsest-balancing-score lemma, and the weighted-balance identities (Lemma 6.4) are properties of the observed treatment-assignment mechanism: they hold for any binary treatment and require no causal assumptions. The causal results of this chapter — the Propensity Score Theorem, the IPW identification formula, the causal interpretation of matching — additionally rest on the unconfoundedness assumption: \[\bigl(Y(0),\, Y(1)\bigr) \;\indep\; T \;\Big|\; X,\] together with consistency and the relevant positivity condition. Unconfoundedness requires that the observed pretreatment set \(X\) be sufficient for adjustment. In graphical terms: \(X\) blocks every back-door path from \(T\) to \(Y\). This does not require that every latent common cause itself be observed: an observed variable lying on a confounding path can block that path even when the common cause behind it is latent (Chapter 3). What is required is that every noncausal path be intercepted by the observed set — and this can never be verified from the data alone. Balance on \(X\) is necessary for a good design and says nothing about an unmeasured \(U\).
Two complementary responses. When unconfoundedness is in doubt, two distinct strategies are available and they answer different questions. Sensitivity analysis (Chapter 9) keeps the back-door framework but quantifies how strong an unmeasured confounder would have to be to overturn the conclusion: if the answer is “implausibly strong” the analysis is robust, whereas if the answer is “modest” the conclusion is fragile. The instrumental variable approach (Chapter 7) replaces the unconfoundedness assumption with a different set of identifying assumptions, exploiting an external source of variation in treatment that is independent of the unobserved treatment–outcome confounders. The hidden-confounding concern is thereby relocated, not eliminated: the instrument itself must be valid — unconfounded with the outcome and excluded from affecting it directly. In short: sensitivity analysis preserves the original analysis and reports a robustness margin; instrumental variables substitute a different identifying assumption. The remainder of this section sketches the second route, since it is the topic of the next chapter.
6.8.2 The Road to Instrumental Variables
The limitation of back-door adjustment motivates the search for alternative identification strategies when unobserved confounders are present. One canonical alternative — the one taken up next — is the instrumental variable (IV), studied in Chapter 7. It is not the only one: front-door identification (Chapter 3) and sensitivity analysis with partial identification (Chapter 9) are others.
| Strategy | Key assumption | Identification mechanism |
|---|---|---|
| Back-door adjustment (propensity score) | Observed pretreatment set blocks all back-door paths | Condition on \(X\) to block \(T \leftarrow U \rightarrow Y\) |
| Instrumental variables | Some confounders may be unobserved | Exploit exogenous variation \(Z \rightarrow T\) |
Back-door adjustment removes confounding by conditioning on observed variables. Instrumental variables take a different approach: instead of blocking the confounding path \(T \leftarrow U \rightarrow Y\), they isolate variation in \(T\) induced by an external source \(Z\) that is independent of \(U\). Under additional IV assumptions, the ratio of the \(Z\)-induced change in \(Y\) to the \(Z\)-induced change in \(T\) — the Wald estimator — identifies a causal parameter. In the binary instrument and treatment setting, under the additional monotonicity (no-defiers) assumption, that parameter is the local average treatment effect (LATE) among compliers. This is derived formally in Chapter 7.
6.9 Summary
The propensity score reduces dimension. The propensity score \(\pi(X) = P(T{=}1 \mid X)\) is a balancing score — \(T \indep X \mid \pi(X)\) — for any treatment-assignment mechanism, with no causal assumption required, and it is the coarsest one. Under unconfoundedness the balancing property upgrades to \((Y(0),Y(1)) \indep T \mid \pi(X)\) (Theorem 6.2), and positivity transfers separately, so identification requires adjustment for the scalar \(\pi(X)\) alone, not the full vector \(X\). Effects conditional on the score are identified; regressing \(Y\) on \(T\) and \(\pi(X)\) is not an implementation of the formula.
Three uses of one scalar. Stratify (Section 6.2.3), weight (Section 6.5), match (Section 6.4). Each implements the back-door formula; they differ in estimand targeting and finite-sample behavior. The nearest-neighbor matching estimator presented here targets the ATT; stratum contrasts weighted by \(n_j/n\) target the ATE and by \(n_{1j}/n_1\) the ATT. The identifying condition for matching depends on the score only through its level sets, so matching can remain consistent, under regularity conditions, when a misspecified fit keeps the level sets of a balancing or a prognostic score; the finite-sample matches and calipers still depend on the score’s scale. IPW uses the score’s values, and wrong values generally give wrong weights.
The outcome-side mirror. A prognostic score \(\Psi(X)\), with \(Y(0) \indep X \mid \Psi(X)\), is the mirror image of a balancing score: the conditional law of \(Y(0)\) is the coarsest one, and ignorability given \(X\) transfers to \(\Psi(X)\) (Theorem 6.3). The mirror breaks where \(T\) and \(Y(0)\) differ: positivity transfers only one way, a prognostic score serves the ATT unless it is also sufficient for \(Y(1)\), and it is fitted on controls with their outcomes.
IPW identification and the two estimators. Under consistency, weak conditional exchangeability for both treatment levels, positivity, and integrability, the ATE is identified as \(\E[TY/\pi(X)] - \E[(1-T)Y/(1-\pi(X))]\) (Theorem 6.4); the weights re-create the target population’s covariate law (Lemma 6.4). The Horvitz–Thompson estimator is the sample analog and need not be location invariant; the Hájek estimator is, which removes the level effect \(c(w_T - w_C)\) but not extreme weights.
A well-estimated score is a control variate. Under a correct regular ML fit, the influence function of IPW is \(\varphi - \Pi(\varphi \mid S)\), so the limiting variance can only fall (Theorem 6.5); the saturated score gives exact cell balance and HT \(=\) Hájek \(=\) standardization. The calibration loss of Tan (2020) is the dual of the shifted Kullback–Leibler calibration problem of Wang and Kim (2025); it fits the logistic score so that the weights balance \(f(X)\) exactly, and exact balance makes the IPW estimate consistent under a linear outcome model even if the score model is wrong. For the ATT the Kullback–Leibler entropy calibrates the controls to the treated totals with odds weights: this is entropy balancing, and its dual is the control-arm loss of the same fit.
Positivity is necessary; strong overlap, with outcome moments, is a standard sufficient condition for stable estimation. Positivity (\(0 < \pi(X) < 1\) a.s.) is necessary for identification of the ATE; a violation on a null set is harmless, and a small positive score is an estimation problem, not an identification failure. Strong overlap (\(c \le \pi(X) \le 1-c\)) together with finite outcome second moments supports the standard \(\sqrt{n}\) theory. Near-violations create extreme IPW weights and large variance; trimming and re-targeting buy stability by changing the question, and clipping the score is neither.
The fundamental limitation. Unconfoundedness \((Y(0),Y(1)) \indep T \mid X\) is a strong, untestable assumption. When unobserved confounders are present, propensity score adjustment fails. An instrumental variable is one important alternative identification strategy (Chapter 7); others include front-door identification (Chapter 3) and sensitivity analysis with partial identification (Chapter 9).
6.10 Problems
- Propensity score and balancing. Suppose \(X = (X_1, X_2)\), where \(X_1\) is binary and \(X_2\) is continuous. The propensity score model is correctly specified as \(\mathrm{logit}(\pi(X)) = \beta_0 + \beta_1 X_1 + \beta_2 X_2\).
- State and prove the Balancing Property \(T \indep X \mid \pi(X)\).
- Two units \(i\) and \(j\) have \(X_i = (1, 2.3)\) and \(X_j = (0, 3.5)\) but \(\pi(X_i) = \pi(X_j) = 0.4\) under the true propensity score. Explain what equality of the true propensity score implies about the conditional distribution of \(X\) across treatment arms at this score value, and why this is a population-level balancing statement rather than an exact individual counterfactual equality: if \(i\) is treated and \(j\) is control, \(Y_j\) is a noisy estimate of a conditional mean, not of \(Y_i(0)\). State the additional assumptions needed for propensity-score matching to consistently estimate the ATT.
- Explain why matching on \(\hat\pi(X)\) is not the same as matching on \(X\) directly. Under what conditions do they give the same answer?
- ATE, ATT, and propensity score weighting. A dataset has \(n = 1000\) observations. The estimated propensity scores are \(\hat\pi(X_i)\) for \(i = 1, \ldots, n\), with \(n_1 = 400\) treated units.
- Write the Horvitz–Thompson IPW estimator of the ATE as a weighted sum of observed outcomes \(Y_i\). What weights do treated units receive? What weights do control units receive?
- Derive an analogous IPW estimator for the ATT. (Hint: the ATT averages over the treated distribution; the denominator of the weight for control units should reflect this.)
- Suppose the propensity score is constant: \(\pi(X) = p\) for all units. Show that the Horvitz–Thompson IPW estimator of the ATE equals \[\frac{n_1}{np}\,\bar Y_1 - \frac{n_0}{n(1-p)}\,\bar Y_0,\] where \(\bar Y_1\) and \(\bar Y_0\) are the arm-specific sample means, and conclude that it reduces to the difference in means \(\bar Y_1 - \bar Y_0\) only when the realized count satisfies \(n_1 = np\) exactly — as under complete randomization with fixed arm sizes — whereas under Bernoulli assignment \(n_1\) is random and the equality fails. Show that the Hájek estimator reduces to \(\bar Y_1 - \bar Y_0\) exactly whenever the score is constant. Under which experimental designs does \(\pi(X) = 0.5\) hold for each unit: Bernoulli randomization with \(P(T_i{=}1) = 0.5\), complete randomization with exactly \(n/2\) treated, or both?
- Overlap. Let \(\pi(X)\) be the true propensity score and suppose overlap fails on a set \(\mathcal{S}_1\) of covariate values with \(P(X \in \mathcal{S}_1) > 0\): \(\pi(x) = 1\) for all \(x \in \mathcal{S}_1\). (For a continuous covariate, a violation at a single \(P_X\)-null point would be harmless; positivity is a statement about non-null sets.)
- For each of \(\tau_{\mathrm{ATE}}\) and \(\tau_{\mathrm{ATT}}\), identify the exact formula that fails to be identified, and explain why. (Hint: consider whether \(\mathcal{S}_1\) intersects the support of \(X \mid T{=}1\).)
- Suppose overlap fails only on a set \(\mathcal{S}\) of covariate values with \(P(X \in \mathcal{S}) = 0.15\). Define the trimmed ATE estimand. How does it differ from the ATE?
- A researcher applies the Crump et al. rule with \(\eta = 0.1\), removing all units with \(\hat\pi(X_i) < 0.1\) or \(\hat\pi(X_i) > 0.9\). This removes 8% of the sample. (i) List two reasons the trimmed estimator may have lower variance than the untrimmed IPW estimator. (ii) What is the cost in terms of external validity?
- Now suppose instead that \(\pi(x)\) is small but strictly positive on \(\mathcal{S}\). Which of the ATE, ATT, and ATC are identified by adjustment? What is the statistical problem, and which condition of Definition 6.5 can fail even though positivity holds?
- Covariate selection for the propensity model. A researcher has pretreatment covariates \(C\), \(P\), and \(Z\), and a post-treatment variable \(M\). In the assumed causal graph, \(C\) is a confounder of \(T\) and \(Y\); \(P\) predicts the outcome but not treatment; \(Z\) strongly predicts treatment but has no causal relation to the potential outcomes; and \(M\) is a mediator on the path from \(T\) to \(Y\).
- Which variables belong in the propensity model when the target is the total effect of \(T\) on \(Y\)? Justify each inclusion or exclusion using the taxonomy of Section 6.3.1.
- Explain how including \(Z\) may affect overlap, the distribution of the inverse-probability weights, and the effective sample size, even though it does not invalidate the adjustment set.
- Suppose the researcher discovers that \(C\) was in fact measured after treatment for a subset of units. What concerns does this raise for the propensity design?
- Rerun the script of Example 6.3 for \(\gamma \in \{0, 0.05, 0.1, 0.2\}\). For which \(\gamma\) does including \(X_2\) give the smaller RMSE? Find approximately the \(\gamma\) at which the two RMSEs are equal, and explain the answer with the bias–variance trade-off of Section 6.3.1.
- Hidden confounding (conceptual preview of Chapter 9). Suppose you estimate \(\hat\tau_{\mathrm{ATE}} = 3.2\) using propensity-score methods, and a referee questions whether unconfoundedness holds.
- Define what it means for \(U\) to be a “hidden confounder” in the context of the DAG, and explain why its presence invalidates the IPW identification formula.
- Explain in words what it means for an estimated effect to be “robust to hidden confounding,” and why such an assessment is necessarily comparative: it depends on a quantitative yardstick for how strong an unmeasured confounder would have to be to overturn the conclusion. Without doing formal calculations, sketch the narrative a researcher might construct to argue that \(\hat\tau_{\mathrm{ATE}} = 3.2\) would survive any plausible hidden confounder. (Chapter 9 develops three formal yardsticks for this purpose: Rosenbaum’s \(\Gamma\), the E-value, and the marginal sensitivity model.)
- How can a valid instrumental variable identify a causal parameter despite unmeasured confounding between \(T\) and \(Y\) (Chapter 7)? State the assumptions that replace treatment–outcome unconfoundedness, and explain why unmeasured confounding between the instrument and the outcome would still invalidate the IV design.
- Matching vs. IPW. You have \(n = 500\) observations with a binary treatment and a scalar propensity score. You compare two estimators: (A) Hájek IPW estimator targeting the ATE, and (B) 1:1 nearest-neighbor matching designed to target the ATT.
- Explain in one sentence why estimator (A) and estimator (B) target different estimands even though both use \(\hat\pi(X)\).
- In your matched sample, the standardized mean difference (SMD) for covariate \(X_1\) is 0.05 after matching and 0.45 before matching. What does this tell you about the success of matching, and what assumption does balance on observed covariates not verify?
- Under what condition on the treatment effect heterogeneity are the ATE and ATT equal?
- Suppose the true score is \(\pi(X) = F(\beta^\top X)\) with an unknown increasing link \(F\), \(X\) is Gaussian, and you fit a logistic model. Using Lemma 6.3 and the result of Li and Duan (1989), explain why matching on the fitted score can be consistent for the ATT, under the regularity conditions of the warning on inference after matching, while the Hájek ATT estimator need not be. Show that the Hájek ATT estimator is exactly unchanged when the fitted logit differs from the true logit by a constant. Rerun design A of Example 6.6 with a probit link and compare the biases.
- The running example. Use the population of Example 6.1.
- Verify \(\tau_{\mathrm{ATE}} = 2.4\), \(\tau_{\mathrm{ATT}} = 30/11\), and the naive contrast \(4.48\). Show that the two cells with \(\pi = 0.5\) have identical covariate distributions in the two arms when pooled, and compute \(m_1(0.5)\) and \(m_0(0.5)\).
- Compute the IPW weights \(1/\pi(x)\) and \(1/\{1-\pi(x)\}\) for each cell and verify that the weighted treated and control arms each reproduce the population cell counts. Verify \(\E\{TY/\pi(X)\} = 7.6\) and \(\E\{(1-T)Y/(1-\pi(X))\} = 5.2\).
- Compute the population least-squares coefficient of \(T\) in the regression of \(Y\) on \((1, T, \pi(X))\) and confirm that it equals \(114/49\). Explain in one sentence why it differs from the ATE.
- Simulate \(n = 500\) units from this population with \(T_i \sim \mathrm{Bernoulli}\{\pi(X_i)\}\) and \(Y_i = \mu(T_i, X_i) + \varepsilon_i\), \(\varepsilon_i \sim \mathrm{N}(0,1)\). Over \(2{,}000\) replications, evaluate the HT and Hájek estimators of the ATE with the true score and with the cell-proportion (saturated) score, on the same data with outcomes \(Y\) and \(Y + 8\). Report the Monte Carlo standard deviations and explain each pattern you see using Equation 6.4 and the remark on the saturated score.
- The calibration loss as a dual problem. Let \(S = \{i : T_i = 1\}\), \(f_i = (1, X_i^\top)^\top\), and \(G(\omega) = (\omega - 1)\{\log(\omega - 1) - 1\}\) for \(\omega > 1\).
- Form the Lagrangian of Equation 6.5 with multiplier \(\lambda\) and show that the stationarity condition in \(\omega_i\) gives \(\omega_i = 1 + \exp(\lambda^\top f_i)\).
- Substitute these weights into the Lagrangian and show that, with \(\gamma = -\lambda\), the dual objective is \(\ell_{\mathrm{CAL}}(\gamma)\) of Equation 6.6, up to sign. Verify directly that \(\partial\ell_{\mathrm{CAL}}/\partial\gamma = 0\) is the calibration constraint \(\sum_i T_i f_i/\hat\pi_i = \sum_i f_i\).
- Explain why \(\sum_i T_i/\hat\pi_i = n\) at the solution and hence why the Horvitz–Thompson and Hájek estimators of \(\E\{Y(1)\}\) coincide. Write down the analogous loss for the control arm and its first-order condition.
- Replace the objective of Equation 6.5 by \(\sum_{i\in S}\omega_i^2/2\), repeat (a)–(b), and identify the resulting weights with the regression weights of Chapter 5 (remark on generalized entropy calibration).
- Prove Proposition 6.1 in detail for the shifted Kullback–Leibler entropy, stating the moment and convergence conditions you use.
- The ATT. For Equation 6.7, derive the weights and the dual Equation 6.8, and show that \(\sum_{i\in S_0}\hat\omega_i = n_1\). Show that the fitted \(\hat\gamma\) coincides with the minimizer of the control-arm loss of part (c), and that \(1/(1-\hat\pi_i) = 1 + \hat\omega_i\) on the controls.
- With a scalar \(X\) and \(f = (1, X)^\top\), suppose every treated \(X_i\) exceeds every control \(X_j\). Show that the dual Equation 6.8 is unbounded below. (Hint: evaluate it along \(\lambda = s(-\max_{j\in S_0} X_j, 1)^\top\) as \(s \to \infty\).) Interpret the result as an overlap failure for the ATT.
- Clipping, trimming, and infinite moments.
- Verify the probability limits \(10/11\) and \(10/21\) in the remark on clipping. Compute the same limits for the trimmed estimand that drops the \(x = 0\) units from both arms, and explain why trimming gives a causal effect in a stated subpopulation while clipping does not.
- In the Part 2 design of the lab, show that \(\E\{\exp(0.5X_1^2 + 0.8X_1)\} = \infty\) when \(X_1 \sim \mathrm{N}(0,1)\), and conclude that the moment condition Equation 6.9 fails although \(0 < \pi^*(X) < 1\) everywhere. For which values of the quadratic coefficient \(a\) in \(\ell(X) = -0.5 + 0.8X_1 + aX_1^2 + \cdots\) is the moment finite?
- Rerun the true-score HT-ATE estimator of Part 2 at \(n = 1{,}000\), \(4{,}000\), and \(16{,}000\) with \(1{,}000\) replications each, and report \(\sqrt{n}\) times the Monte Carlo standard deviation. Explain what you observe in the light of Definition 6.5.
- Prognostic scores.
- Prove Lemma 6.2 and Theorem 6.3 (i), and write the proof of Theorem 6.2 next to the latter, line by line.
- Show that \(P\{T{=}1 \mid \Psi(X)\} = \E\{\pi(X) \mid \Psi(X)\}\), and conclude that positivity given \(X\) implies positivity given \(\Psi(X)\) but not conversely. Why does the converse hold for a balancing score?
- Let \(X\) be uniform on \(\{1, \ldots, 6\}\) with \(\pi = (1, 1/5, 1/2, 3/4, 1/4, 3/5)\). Let \(Y(0)\) take the values \(2\) and \(4\) with probability \(1/2\) each when \(X \in \{1, 2\}\), the values \(5\) and \(7\) with probability \(1/2\) each when \(X \in \{3, 4\}\), and the value \(6\) when \(X \in \{5, 6\}\), and let \(Y(1) = (9, 4, 10, 8, 7, 12)\) at \(X = 1, \ldots, 6\), with \(Y(0) \indep T \mid X\). Verify that \(\Psi(X)\), the indicator of the three pairs, is a prognostic score, that the ATT equals \(271/66\), and that the formula of Theorem 6.3 (ii) recovers it although the ATT is not identified by adjustment for \(X\). Verify that stratifying on \(m_0(X)\) alone also recovers it, although \(m_0(X)\) is not a prognostic score here.