10 Estimating Equations and Influence Functions
10.1 Why Estimation Needs Its Own Theory
So far the focus has been on identification: under what assumptions can a causal parameter be written as a functional of the observed-data distribution? Identification does not by itself provide a statistically reliable estimator. Once a causal parameter is identified, several distinct questions remain:
- How should the estimator be constructed from the observed sample?
- How does estimation of nuisance functions affect the target estimator?
- What is the large-sample distribution of the estimator?
- How can we compute valid standard errors and confidence intervals?
These questions motivate a separate theory of estimation and inference. In causal inference this issue is especially important because identified parameters often depend on auxiliary, or nuisance, functions such as the outcome regression \(\mu_t(x) = \E(Y \mid T=t,\, X=x)\) or the propensity score \(\pi(x) = P(T=1 \mid X=x)\). Even when the causal parameter is identified, different estimation strategies may behave quite differently in terms of robustness, efficiency, and sensitivity to model misspecification.
The goal of this chapter is to introduce a general framework for estimation based on estimating equations and influence functions. This framework will serve as the foundation for the doubly robust and semiparametric methods developed in Chapter 11; see also Imbens and Rubin (2015) and Hernán and Robins (2020) for complementary treatments.
10.2 A Running Example: The Average Treatment Effect
Throughout this chapter we use the average treatment effect (ATE) as a running example. Let \(\tau = \E\{Y(1) - Y(0)\}\). Whenever the ATE is invoked, we assume the arm-specific potential outcomes are integrable, \(\E|Y(t)| < \infty\) for \(t = 0, 1\), so that \(\tau\) is well defined; additional second-moment conditions are stated where asymptotic normality is required. Under consistency, conditional exchangeability, and positivity, the ATE is identified by the back-door formula (Chapter 5), \[\tau = \E[\mu_1(X) - \mu_0(X)], \qquad \mu_t(x) = \E(Y \mid T=t,\, X=x),\quad t\in\{0,1\}.\]
This identity tells us what the target parameter is, but it does not uniquely determine how to estimate it. For example, one may consider a regression-based estimator obtained by fitting models for \(\mu_1(x)\) and \(\mu_0(x)\); a weighting estimator based on the propensity score \(\pi(x)\) (Chapter 6); or an augmented estimator combining both outcome and treatment models. All of these estimators may target the same causal parameter, yet they differ in their statistical properties. A main goal of this chapter is to develop a common language for describing and comparing such estimators.
10.3 Estimating Equations
A broad class of estimators can be defined as solutions to estimating equations. Let \(O_1,\dots,O_n\) be i.i.d. observations from a distribution \(P\), and let \(\theta\) denote a finite-dimensional target parameter. Write \(\mathbb{P}_n f = n^{-1}\sum_{i=1}^n f(O_i)\) for the empirical average.
An estimating-equation estimator \(\hat\theta\) is any solution to the sample moment condition above. Many familiar estimators take this form.
Example 10.1 (Ordinary Least Squares) Let \(O = (X,Y)\) with \(\E\|X\|^2 < \infty\), \(\E(Y^2) < \infty\), and \(\E(XX^\top)\) nonsingular, and suppose \(\beta\) is defined by the linear moment condition \(\E[X\{Y - X^\top\beta\}] = 0\); the nonsingularity makes \(\beta\) unique. If an intercept is intended, \(X\) is understood to include a constant component. Then the OLS estimator solves \[\mathbb{P}_n\bigl[X\{Y - X^\top\beta\}\bigr] = 0,\] which has a unique solution once the sample matrix \(\mathbb{P}_n(XX^\top)\) is invertible, an event of probability tending to one. For the central limit theorem applied to this estimating function we additionally require \(\E\|X(Y - X^\top\beta)\|^2 < \infty\).
Example 10.2 (A Simple IPW Estimating Equation) Suppose the propensity score \(\pi(X)\) is known. Under the identification assumptions of consistency, conditional exchangeability, and positivity (Chapter 5), the ATE \(\tau = \E\{Y(1) - Y(0)\}\) admits the inverse-probability representation \[\tau = \E\left\{\frac{TY}{\pi(X)} - \frac{(1-T)Y}{1-\pi(X)}\right\}.\] Then \(\tau\) solves \(\E[TY/\pi(X) - (1-T)Y/\{1-\pi(X)\} - \tau] = 0\), and the corresponding estimator replaces the expectation by the empirical average. The right-hand side is an observed-data functional only after identification has been imposed; without those assumptions, knowing \(\pi(X)\) alone does not turn the expression into a causal quantity.
A word about integrability, because weak positivity is treacherous here. Under consistency, conditional exchangeability, positivity, and the standing assumption \(\E|Y(t)| < \infty\), the IPW moment is well defined and equals \(\tau\); equivalently, one may impose the observed-data condition \[\E\!\left[\frac{T|Y|}{\pi(X)} + \frac{(1-T)|Y|}{1-\pi(X)}\right] < \infty .\] The observed second moment \(\E(Y^2) < \infty\) does not by itself deliver this: treatment may be rare precisely where the treated outcome is large, so that large values contribute little to the observed moment while being magnified by \(1/\pi(X)\). A square-integrable influence function requires the stronger weighted-second-moment condition \[\E\!\left[\frac{TY^2}{\pi(X)^2} + \frac{(1-T)Y^2}{\{1-\pi(X)\}^2}\right] < \infty ,\] which is what the asymptotic theory of Section 10.4 needs. Strong overlap together with \(\E(Y^2) < \infty\) is a convenient sufficient condition for the latter; see Section 10.9.1 for the precise requirement.
The main advantage of the estimating-equation framework is that it provides a unified language for both estimator construction and asymptotic analysis.
10.4 From Estimating Equations to Asymptotic Linearity
Estimating equations are useful not only because they define estimators, but also because they often yield a convenient first-order expansion. Under suitable regularity conditions, an estimator solving an estimating equation can typically be approximated as \[\sqrt{n}(\hat\theta - \theta_0) = \frac{1}{\sqrt{n}}\sum_{i=1}^n \varphi(O_i) + o_p(1)\] for some mean-zero function \(\varphi(O)\).
Definition 10.1 (Asymptotic Linearity and Influence Function) An estimator \(\hat\theta\) is asymptotically linear with influence function \(\varphi(O)\) if \[\sqrt{n}(\hat\theta - \theta_0) = \frac{1}{\sqrt{n}}\sum_{i=1}^n \varphi(O_i) + o_p(1), \qquad \E\{\varphi(O)\} = 0,\] where \(\varphi \in L_2^0(P)\), that is, \(\E\|\varphi(O)\|^2 < \infty\).
The square-integrability requirement is what makes the representation useful: it is exactly the condition under which the central limit theorem applies to the leading sum.
This representation is fundamental because it immediately implies asymptotic normality by the multivariate central limit theorem. When \(\theta \in \mathbb{R}^p\) the influence function \(\varphi(O)\) is also \(\mathbb{R}^p\)-valued, and \[\sqrt{n}(\hat\theta - \theta_0) \overset{d}{\longrightarrow} N\!\left(\mathbf{0},\; \Sigma\right), \qquad \Sigma = \E\bigl[\varphi(O)\varphi(O)^\top\bigr],\] where \(\Sigma\) is the \(p \times p\) asymptotic covariance matrix. In the scalar case \(p = 1\) this reduces to \(\Sigma = \E[\varphi(O)^2]\). Once the influence function is identified, we obtain both the asymptotic distribution and a principled basis for variance estimation and confidence intervals.
Proposition 10.1 (Generic First-Order Expansion) Under smoothness and regularity conditions, an estimator \(\hat\theta\) solving \(\mathbb{P}_n\{U(O;\theta)\}=0\) admits a first-order expansion \[\sqrt{n}(\hat\theta - \theta_0) = -A^{-1}\,\frac{1}{\sqrt{n}}\sum_{i=1}^n U(O_i;\theta_0) + o_p(1),\] where \(A = \E\{\partial_{\theta^\top}U(O;\theta_0)\}\). Hence the influence function is \(\varphi(O) = -A^{-1}U(O;\theta_0)\).
Proof. A Taylor expansion of the sample moment condition around \(\theta_0\) gives \[0 = \mathbb{P}_n\{U(O;\hat\theta)\} \approx \mathbb{P}_n\{U(O;\theta_0)\} + \mathbb{P}_n\!\left\{\partial_{\theta^\top} U(O;\theta_0)\right\}(\hat\theta - \theta_0).\] By the law of large numbers, \(\mathbb{P}_n\{\partial_{\theta^\top} U(O;\theta_0)\} \to A\) in probability. Rearranging and multiplying by \(\sqrt{n}\) yields the stated expansion. \(\square\)
Proposition 10.1 is a standard Z-estimation result and shows why estimating equations naturally lead to asymptotic linearity.
10.5 Influence Functions: Intuition
10.5.1 Statistical Functionals
Many parameters of interest in statistics and causal inference can be written as functionals of the underlying distribution. We make this precise before defining the influence function.
10.5.2 Influence Functions
Both objects called “influence function” formalize the same intuition: the first-order effect of a single observation.
An influence function records the infinitesimal contribution of one observation to a first-order expansion.
But whose first-order expansion is left open by that sentence, and the two available answers are not interchangeable. One attaches to an estimator, describing how \(\hat\psi\) responds to perturbing the sample; the other attaches to a functional together with a model, describing how \(\Psi(P)\) responds to perturbing the distribution. Conflating them is the most common source of confusion in this material, so we define them separately and then state exactly how they are related.
Definition 10.2 (Influence Function of an Estimator (informal)) Let \(\psi = \Psi(P)\) be a scalar parameter and let \(\hat\psi\) be an estimator of \(\psi\). A mean-zero function \(\varphi \in L_2(P)\) is called the influence function of the estimator \(\hat\psi\) if \(\hat\psi\) is asymptotically linear at \(P\), that is, \[\sqrt{n}(\hat\psi - \psi) = \frac{1}{\sqrt{n}}\sum_{i=1}^n \varphi(O_i) + o_p(1), \qquad \E\{\varphi(O)\} = 0.\] This is a property of the estimator \(\hat\psi\).
This is the same expansion as in Definition 10.1; the unscaled form \(\hat\psi - \psi = n^{-1}\sum_i\varphi(O_i) + o_p(n^{-1/2})\) is equivalent and is also commonly used.
Definition 10.3 (Pathwise Gradient of a Functional (informal)) Let \(\Psi\) be a functional defined on a statistical model \(\mathcal{P}\). A mean-zero function \(\varphi \in L_2(P)\) is called a gradient of \(\Psi\) at \(P\) relative to \(\mathcal{P}\) if, for every regular parametric submodel \(\{P_\epsilon\} \subset \mathcal{P}\) passing through \(P\) at \(\epsilon = 0\) with score \(S\), \[\left.\frac{d}{d\epsilon}\Psi(P_\epsilon)\right|_{\epsilon=0} = \E_P\{\varphi(O)\,S(O)\}.\] This is a property of the parameter \(\Psi\) and the model \(\mathcal{P}\); it makes no reference to any estimator.
The figure below collects this architecture in one place. The left column is the estimator-side theory developed in Section 10.4–Section 10.8: one starts from an estimating function, expands, and reads off a variance. The right column is the functional-side theory of Section 10.9 and Appendix C: one starts from a target functional and a model, differentiates along submodels, and projects. The two columns are developed independently and meet only through the bridge at the bottom. It is worth returning to this figure as each piece is introduced.
The following calculation carries out the contamination heuristic in full for the OLS coefficient. On a first reading, the point to extract is the conclusion — the two routes agree here, and the paragraph headed “What this does and does not establish” explains why that agreement is special to the nonparametric model. The intervening matrix-derivative algebra can be deferred.
Example 10.3 (OLS Slope by Contamination Derivative) Second reading.
Let \(O = (X, Y) \in \mathbb{R}^{p+1}\), under the moment and nonsingularity conditions of Example 10.1, and consider the population linear regression coefficient \[\beta = \Psi(P) = \bigl[\E(XX^\top)\bigr]^{-1}\E(XY),\] a nonlinear functional of \(P\). The OLS estimator \(\hat\beta = (\sum_i X_iX_i^\top)^{-1}\sum_i X_iY_i\) is the empirical plug-in \(\Psi(\mathbb{P}_n)\).
Step 1: Construct the perturbed distribution. Let \(P_\epsilon = (1-\epsilon)P + \epsilon\delta_O\) be the mixture of \(P\) with a point mass at a fixed observation \(O = (X, Y)\). Then \[\E_{P_\epsilon}(\tilde X \tilde X^\top) = (1-\epsilon)\E(XX^\top) + \epsilon XX^\top, \qquad \E_{P_\epsilon}(\tilde X \tilde Y) = (1-\epsilon)\E(XY) + \epsilon XY,\] so the functional evaluated at \(P_\epsilon\) is \[\Psi(P_\epsilon) = \bigl[(1-\epsilon)\E(XX^\top) + \epsilon XX^\top\bigr]^{-1}\bigl[(1-\epsilon)\E(XY) + \epsilon XY\bigr].\]
Step 2: Differentiate at \(\epsilon = 0\). Write \(M(\epsilon) = (1-\epsilon)\E(XX^\top) + \epsilon XX^\top\) and \(v(\epsilon) = (1-\epsilon)\E(XY) + \epsilon XY\), so \(\Psi(P_\epsilon) = M(\epsilon)^{-1}v(\epsilon)\). Both paths are linear in \(\epsilon\), with derivatives \[\dot M(0) = XX^\top - \E(XX^\top), \qquad \dot v(0) = XY - \E(XY).\]
Deriving the derivative of a matrix inverse. For any differentiable matrix-valued path \(M(\epsilon)\) with \(M(\epsilon)\) invertible, differentiating the identity \(M(\epsilon)M(\epsilon)^{-1} = I\) gives \(\dot M M^{-1} + M \frac{d}{d\epsilon}M^{-1} = 0\), so \[\frac{d}{d\epsilon}M(\epsilon)^{-1} = -M(\epsilon)^{-1}\dot M(\epsilon)\,M(\epsilon)^{-1}.\] Evaluating at \(\epsilon = 0\), with \(M(0) = \E(XX^\top)\), \[\left.\frac{d}{d\epsilon}M(\epsilon)^{-1}\right|_{\epsilon=0} = -\bigl[\E(XX^\top)\bigr]^{-1}\bigl(XX^\top - \E(XX^\top)\bigr)\bigl[\E(XX^\top)\bigr]^{-1}.\]
Applying the product rule. Using \(\frac{d}{d\epsilon}[M^{-1}v] = \dot{(M^{-1})}v + M^{-1}\dot v\), \[\begin{aligned} \varphi(O) &= \left.\frac{d}{d\epsilon}\Psi(P_\epsilon)\right|_{\epsilon=0} \\ &= -\bigl[\E(XX^\top)\bigr]^{-1}\bigl(XX^\top - \E(XX^\top)\bigr)\bigl[\E(XX^\top)\bigr]^{-1}\E(XY) + \bigl[\E(XX^\top)\bigr]^{-1}\bigl(XY - \E(XY)\bigr). \end{aligned}\] Substituting \(\beta = [\E(XX^\top)]^{-1}\E(XY)\) and collecting terms, \[\begin{aligned} \varphi(O) &= \bigl[\E(XX^\top)\bigr]^{-1}\bigl\{XY - \E(XY) - (XX^\top - \E(XX^\top))\beta\bigr\} \\ &= \bigl[\E(XX^\top)\bigr]^{-1}X\bigl(Y - X^\top\beta\bigr) - \bigl[\E(XX^\top)\bigr]^{-1}\bigl\{\E(XY) - \E(XX^\top)\beta\bigr\}. \end{aligned}\] The second term is zero because \(\beta = [\E(XX^\top)]^{-1}\E(XY)\) implies \(\E(XY) - \E(XX^\top)\beta = 0\). Hence \[\varphi(O) = \bigl[\E(XX^\top)\bigr]^{-1}X\bigl(Y - X^\top\beta\bigr).\]
Verification. One can confirm \(\E\{\varphi(O)\} = 0\) directly: \(\E[X(Y - X^\top\beta)] = \E(XY) - \E(XX^\top)\beta = 0\) by definition of \(\beta\).
Connection to the estimating-equation approach. The OLS estimating function is \(U(O;\beta) = X(Y - X^\top\beta)\), so \(A = \E\{\partial_{\beta^\top}U\} = -\E(XX^\top)\) and Proposition 10.1 gives \(\varphi(O) = -A^{-1}U(O;\beta_0) = [\E(XX^\top)]^{-1}X(Y - X^\top\beta)\), agreeing with the contamination calculation.
What this does and does not establish. The two routes answer different questions, and their agreement here is informative rather than automatic. The estimating-equation route delivers the influence function of the OLS estimator (Definition 10.2). The contamination route produces a candidate gradient of the functional \(\Psi\) (Definition 10.3); as noted above, it is a heuristic, and confirming that the candidate is a genuine gradient requires verifying the score identity against regular submodels. In the nonparametric model the tangent space is all of \(L_2^0(P)\), the gradient is unique when it exists, and the two objects coincide — which is exactly why OLS is a clean illustration. In restricted models the two can differ, and the distinction of Definition 10.2 and Definition 10.3 becomes essential.
10.6 Influence Functions for Simple Causal Estimators
We now illustrate the idea in causal settings.
10.6.1 Known Outcome Regression
Suppose \(\tau = \E[\mu_1(X) - \mu_0(X)]\) and the functions \(\mu_1(\cdot)\) and \(\mu_0(\cdot)\) are known. The natural plug-in estimator is \(\hat\tau_{\mathrm{reg}} = \mathbb{P}_n\{\mu_1(X) - \mu_0(X)\}\). Its influence function, which we denote \(\varphi_{\mathrm{reg}}\), is \[\varphi_{\mathrm{reg}}(O) = \mu_1(X) - \mu_0(X) - \tau . \tag{10.1}\] Note that \(\varphi_{\mathrm{reg}}\) depends on \(O\) only through \(X\).
10.6.2 Known Propensity Score
Suppose \(\pi(X)\) is known and \(\tau = \E\{TY/\pi(X) - (1-T)Y/[1-\pi(X)]\}\). The empirical analogue is \[\hat\tau_{\mathrm{IPW}} = \mathbb{P}_n\!\left\{\frac{TY}{\pi(X)} - \frac{(1-T)Y}{1-\pi(X)}\right\},\] and its influence function, denoted \(\varphi_{\mathrm{IPW}}\), is \[\varphi_{\mathrm{IPW}}(O) = \frac{TY}{\pi(X)} - \frac{(1-T)Y}{1-\pi(X)} - \tau . \tag{10.2}\] These two symbols are used throughout the remainder of the chapter and in the problems; both are defined only under the oracle conventions stated above.
10.7 Z-Estimation with Nuisance Parameters
The examples in Section 10.6 treated nuisance functions such as \(\mu_t(x)\) and \(\pi(x)\) as known. That simplification was pedagogically useful because it made the influence function transparent, but it is not realistic in practice. In real applications, nuisance functions must be estimated from the same data used to estimate the target parameter. Once this happens, the first-order expansion of \(\hat\psi\) is no longer obtained by simply centering the estimating function with the nuisance held fixed: the randomness in the estimated nuisance introduces an additional term that the naive plug-in formula ignores. We therefore need a framework that accounts jointly for estimation of the target parameter and the nuisance parameter. Z-estimation provides that framework: it treats both as components of a single stacked estimating equation, so the corrected influence function — including the nuisance-estimation term — falls out of one unified first-order expansion.
10.7.1 The Stacked Estimating Equation Framework
The key idea is that once \(\alpha\) is estimated rather than known, the target equation for \(\psi\) and the nuisance equation for \(\alpha\) must be analyzed jointly rather than one after the other: only the joint analysis reveals the additional term that the estimated nuisance contributes to the first-order expansion of \(\hat\psi\).
Suppose the target parameter \(\psi \in \mathbb{R}^p\) depends on an unknown nuisance parameter \(\alpha \in \mathbb{R}^q\) (e.g., regression coefficients in a parametric model for \(\pi(x)\)). Assume both are estimated by solving a stacked system of estimating equations: \[\mathbb{P}_n\bigl\{U_1(O;\psi,\alpha)\bigr\} = 0, \tag{10.3}\] \[\mathbb{P}_n\bigl\{U_2(O;\alpha)\bigr\} = 0. \tag{10.4}\] Here \(U_1(O;\psi,\alpha) \in \mathbb{R}^p\) defines the target estimator \(\hat\psi\) given \(\alpha\), while \(U_2(O;\alpha) \in \mathbb{R}^q\) defines the nuisance estimator \(\hat\alpha\). These are solved simultaneously (or sequentially: first Equation 10.4 for \(\hat\alpha\), then Equation 10.3 for \(\hat\psi\)). In the running ATE example, \(p = 1\) and \(\psi = \tau\), so \(U_1\) is scalar.
10.7.2 The Z-Estimation Theorem
Write \(\theta = (\psi^\top, \alpha^\top)^\top \in \mathbb{R}^{p+q}\) and let \[U(O;\theta) = \begin{pmatrix} U_1(O;\psi,\alpha) \\ U_2(O;\alpha) \end{pmatrix}\] be the stacked estimating function. The joint estimator \(\hat\theta = (\hat\psi^\top, \hat\alpha^\top)^\top\) solves \(\mathbb{P}_n\{U(O;\hat\theta)\} = 0\).
Theorem 10.1 (Z-Estimation Theorem (Stacked Equations)) Suppose the true parameter value is \(\theta_0 = (\psi_0^\top,\alpha_0^\top)^\top\) and the following regularity conditions hold:
- \(\E\{U(O;\theta_0)\}=0\) (population moment condition);
- \(\hat\theta \overset{p}{\to} \theta_0\) (consistency of the joint estimator);
- the Jacobian \(\mathcal{A} = \E\{\partial_{\theta^\top} U(O;\theta_0)\}\) is invertible;
- \(U(O;\theta)\) is smooth enough in \(\theta\) that a uniform law of large numbers and a CLT apply.
Then \[\sqrt{n}(\hat\theta - \theta_0) = -\mathcal{A}^{-1}\,\frac{1}{\sqrt{n}}\sum_{i=1}^n U(O_i;\theta_0) + o_p(1).\] In particular, after partitioning \(\mathcal{A}^{-1}\) conformably with \((\psi,\alpha)\) as detailed below, the influence function of \(\hat\psi\) is \[\varphi(O) = -A_{11}^{-1}\,U_1(O;\psi_0,\alpha_0) + A_{11}^{-1}A_{12}A_{22}^{-1}\,U_2(O;\alpha_0).\]
Proof. The argument mirrors Proposition 10.1 applied to the full stacked system. A Taylor expansion of \(\mathbb{P}_n\{U(O;\hat\theta)\}=0\) around \(\theta_0\) gives \[0 \approx \mathbb{P}_n\{U(O;\theta_0)\} + \mathbb{P}_n\!\left\{\partial_{\theta^\top} U(O;\theta_0)\right\}(\hat\theta-\theta_0).\] By the law of large numbers, the matrix of derivatives converges to \(\mathcal{A}\). Rearranging and multiplying by \(\sqrt{n}\), \[\sqrt{n}(\hat\theta - \theta_0) \approx -\mathcal{A}^{-1}\,\frac{1}{\sqrt{n}}\sum_{i=1}^n U(O_i;\theta_0).\] Extracting the first component gives the influence function of \(\hat\psi\). \(\square\)
To see the structure explicitly, partition \(\mathcal{A}\) conformably with \((\psi,\alpha)\): \[\mathcal{A} = \begin{pmatrix} A_{11} & A_{12} \\ 0 & A_{22} \end{pmatrix},\] where \(A_{11} = \E\{\partial_{\psi^\top} U_1\}\), \(A_{12} = \E\{\partial_{\alpha^\top} U_1\}\), and \(A_{22} = \E\{\partial_{\alpha^\top} U_2\}\). The lower-left block is zero because \(U_2\) does not depend on \(\psi\). By block-matrix inversion, \[\mathcal{A}^{-1} = \begin{pmatrix} A_{11}^{-1} & -A_{11}^{-1}A_{12}A_{22}^{-1} \\ 0 & A_{22}^{-1} \end{pmatrix}.\] Reading off the first row, the influence function of \(\hat\psi\) is \[\varphi(O) = -A_{11}^{-1}\,U_1(O;\psi_0,\alpha_0) + A_{11}^{-1}A_{12}A_{22}^{-1}\,U_2(O;\alpha_0). \tag{10.5}\]
10.7.3 A Simple Stacked Example: A Ratio of Means
Before applying Equation 10.5 to the propensity-score problem, it is worth seeing the block correction operate in a setting with no causal structure and no matrix algebra, where every quantity can be computed in one line.
Example 10.4 (Ratio of Means) Let \(O = (Y, Z)\) with \(\E(Z) \neq 0\) and finite second moments, and let the target be the ratio \(\psi = \E(Y)/\E(Z)\). Treat \(\alpha = \E(Z)\) as a nuisance parameter and estimate \((\psi,\alpha)\) jointly from the stacked system \[U_1(O;\psi,\alpha) = Y - \psi\alpha, \qquad U_2(O;\alpha) = Z - \alpha .\] Both population moment conditions hold at \((\psi_0,\alpha_0) = (\E Y/\E Z,\ \E Z)\).
Jacobian blocks. Differentiating, \[A_{11} = -\alpha_0 = -\E(Z), \qquad A_{12} = -\psi_0, \qquad A_{22} = -1 .\] Note \(A_{12} = -\psi_0 \neq 0\) whenever \(\psi_0 \neq 0\), so the nuisance correction genuinely matters here.
Influence function. Substituting into Equation 10.5, \[\varphi(O) = -A_{11}^{-1}U_1 + A_{11}^{-1}A_{12}A_{22}^{-1}U_2 = \frac{Y - \psi_0\E(Z)}{\E(Z)} - \frac{\psi_0\{Z - \E(Z)\}}{\E(Z)} ,\] and the terms in \(\psi_0\E(Z)\) cancel, leaving the compact form \[\varphi(O) = \frac{Y - \psi_0 Z}{\E(Z)}.\] One checks directly that \(\E\{\varphi(O)\} = \{\E(Y) - \psi_0\E(Z)\}/\E(Z) = 0\), as required.
What to notice. Had we ignored the nuisance and simply centered \(U_1\) at the true \(\alpha_0\), we would have obtained \(\{Y - \psi_0\E(Z)\}/\E(Z)\), which involves only the randomness in \(Y\). Call that oracle influence function \(\varphi_{\mathrm{orc}}(O)\). The correction term supplies the \(-\psi_0 Z\) piece, capturing the fact that \(\hat\alpha = \bar Z\) is itself estimated.
Whether this raises or lowers the asymptotic variance is not settled by signs alone. Comparing the two, \[\mathrm{Var}\{\varphi(O)\} - \mathrm{Var}\{\varphi_{\mathrm{orc}}(O)\} = \frac{\psi_0\{\psi_0\mathrm{Var}(Z) - 2\,\mathrm{Cov}(Y,Z)\}}{\{\E(Z)\}^2},\] so for \(\psi_0 > 0\) the correction reduces the variance exactly when \(\psi_0\mathrm{Var}(Z) < 2\,\mathrm{Cov}(Y,Z)\). Positive correlation between \(Y\) and \(Z\) is therefore not sufficient: if \(\mathrm{Cov}(Y,Z)\) is positive but small relative to \(\psi_0\mathrm{Var}(Z)\), estimating \(\alpha\) increases the asymptotic variance.
This is worth dwelling on, because the propensity-score example of the next subsection behaves differently. There the correction can only reduce the variance, and that guarantee is not a general feature of the block formula: it comes from \(U_2\) being a maximum likelihood score for a correctly specified model, which supplies the Bartlett and cross identities Equation 10.8 and Equation 10.9. Those identities fail here — \(A_{12} = -\psi_0\) while \(\E[U_1U_2] = \mathrm{Cov}(Y,Z)\), so \(A_{12} = -\E[U_1U_2]\) only in the coincidental case \(\psi_0 = \mathrm{Cov}(Y,Z)\). Absent that structure, the nuisance correction is simply a correction: it can move the variance in either direction.
10.7.4 Working Example: IPW with Estimated Propensity Score
Chapter 6 introduced IPW as an estimator motivated by the back-door identification formula and the propensity score. The present example revisits the same estimator from a different angle: not identification, but first-order asymptotic analysis when the propensity score is itself estimated. The generic target parameter \(\psi\) is now the ATE \(\tau\).
Setup. Assume a logistic regression model \(\pi(X;\alpha) = \expit(X^\top\alpha)\). The IPW estimating equation for \(\tau\) is \[U_1(O;\tau,\alpha) = \frac{TY}{\pi(X;\alpha)} - \frac{(1-T)Y}{1-\pi(X;\alpha)} - \tau,\] and the score equation for the logistic regression is \(U_2(O;\alpha) = X\{T - \pi(X;\alpha)\}\). The stacked system \(\mathbb{P}_n U_1 = 0\), \(\mathbb{P}_n U_2 = 0\) yields \((\hat\tau, \hat\alpha)\).
Computing the Jacobian blocks. The blocks \(A_{11}\) and \(A_{22}\) are immediate: \[A_{11} = -1, \qquad A_{22} = -\E\{\pi(X;\alpha_0)(1-\pi(X;\alpha_0))XX^\top\} =: -\Sigma_\pi.\] For \(A_{12}\), use the logistic score identity \(\partial\pi/\partial\alpha^\top = \pi(1-\pi)X^\top\). Differentiating \(1/\pi\) and \(1/(1-\pi)\) then gives \[\frac{\partial}{\partial\alpha^\top}\!\left[\frac{1}{\pi}\right] = -\frac{1-\pi}{\pi}\,X^\top, \qquad \frac{\partial}{\partial\alpha^\top}\!\left[\frac{1}{1-\pi}\right] = \frac{\pi}{1-\pi}\,X^\top,\] so \[\frac{\partial U_1}{\partial\alpha^\top} = -\,Y X^\top\!\left\{\frac{T(1-\pi)}{\pi} + \frac{(1-T)\pi}{1-\pi}\right\}.\] Using the identities \(T(1-\pi)/\pi = T/\pi - T\) and \((1-T)\pi/(1-\pi) = (1-T)/(1-\pi) - (1-T)\), this simplifies to \[\frac{\partial U_1}{\partial\alpha^\top} = -\,Y X^\top\!\left\{\frac{T}{\pi} + \frac{1-T}{1-\pi} - 1\right\}.\] Evaluating at \(\alpha_0\) and taking expectations, \[A_{12} = -\,\E\!\left[Y X^\top\!\left\{\frac{T}{\pi(X;\alpha_0)} + \frac{1-T}{1-\pi(X;\alpha_0)} - 1\right\}\right].\]
The corrected influence function. Substituting into Equation 10.5 with \(A_{11}=-1\) (so \(-A_{11}^{-1} = 1\)) and \(A_{22}^{-1} = -\Sigma_\pi^{-1}\), the two negative signs cancel in the second term, giving \[\varphi(O) = U_1(O;\tau_0,\alpha_0) + A_{12}\,\Sigma_\pi^{-1}\,U_2(O;\alpha_0). \tag{10.6}\] The first term is the naive IPW influence function from Section 10.6.2. The second term is the nuisance correction: it removes the first-order contribution of \(\hat\alpha - \alpha_0\) to the estimation error in \(\hat\tau\).
The next example works out the consequence of this correction for the asymptotic variance. It is the most technically demanding passage in the chapter, requiring the stacked expansion, the likelihood score, the information equality, and differentiation of an expectation whose law depends on \(\alpha\). On a first reading it is enough to take away the conclusion: estimating a correctly specified parametric propensity score by maximum likelihood can only lower the asymptotic variance of the IPW estimator.
Example 10.5 (Variance of IPW with Estimated Propensity Score) Advanced.
We show that estimating \(\alpha\) by maximum likelihood (ML) can only reduce the asymptotic variance of \(\hat\tau_{\mathrm{IPW}}\) relative to using the true \(\alpha_0\). Two Bartlett-type identities are the key.
Variance of the corrected influence function. Write \(\sigma_{\mathrm{naive}}^2 = \E[U_1(O;\tau_0,\alpha_0)^2]\) for the variance of the naive IPW estimating function when \(\alpha_0\) is treated as known. Expanding the square of Equation 10.6, \[\E[\varphi(O)^2] = \sigma_{\mathrm{naive}}^2 + 2\,A_{12}\Sigma_\pi^{-1}\E[U_1 U_2] + A_{12}\Sigma_\pi^{-1}\E[U_2 U_2^\top]\Sigma_\pi^{-1}A_{12}^\top. \tag{10.7}\]
Identity 1: Bartlett identity for the ML score. Because \(U_2(O;\alpha) = X\{T - \pi(X;\alpha)\}\) is the ML score for \(\alpha\) in a correctly specified logistic model, it satisfies the information equality \[\E\!\left[\partial_{\alpha^\top}U_2(O;\alpha_0)\right] + \E\bigl[U_2 U_2^\top\bigr] = 0.\] Since \(A_{22} = -\Sigma_\pi\), this gives \[\E\bigl[U_2 U_2^\top\bigr] = \Sigma_\pi. \tag{10.8}\]
Identity 2: a cross-identity linking \(A_{12}\) to \(\E[U_1 U_2^\top]\). A different identity constrains \(A_{12}\). It comes not from \(U_1\) being a score (it is not), but from the fact that \(\E_\alpha[U_1]\) is identically zero as a function of \(\alpha\), where \(\E_\alpha\) denotes expectation under a data-generating distribution in which the propensity score is \(\pi(X;\alpha)\) and the distributions \(P_X\) and \(P_{Y\mid T,X}\) are held fixed. Under this convention, and assuming consistency, conditional exchangeability, and correct specification of the propensity model, \[\E_\alpha[U_1(O;\tau_0,\alpha)] = \E_X[\mu_1(X)] - \E_X[\mu_0(X)] - \tau_0 = 0 \quad\text{for all }\alpha.\] Differentiating both sides with respect to \(\alpha^\top\) and applying the Leibniz rule, \[0 = \frac{\partial}{\partial\alpha^\top}\E_\alpha[U_1] = \E_{\alpha}\!\left[\frac{\partial U_1}{\partial\alpha^\top}\right] + \E_{\alpha}[U_1 U_2^\top].\] Evaluating at \(\alpha = \alpha_0\) gives \[A_{12} = -\E[U_1 U_2^\top]. \tag{10.9}\]
Variance reduction. Identity Equation 10.9 gives \(\E[U_1 U_2] = -A_{12}^\top\), so the middle term of Equation 10.7 becomes \(-2 A_{12}\Sigma_\pi^{-1}A_{12}^\top\), while Equation 10.8 collapses the third term to \(A_{12}\Sigma_\pi^{-1}A_{12}^\top\). Hence \[\begin{aligned} \E[\varphi(O)^2] &= \sigma_{\mathrm{naive}}^2 - 2\,A_{12}\Sigma_\pi^{-1}A_{12}^\top + A_{12}\Sigma_\pi^{-1}A_{12}^\top \\ &= \sigma_{\mathrm{naive}}^2 - A_{12}\Sigma_\pi^{-1}A_{12}^\top = \sigma_{\mathrm{naive}}^2 - \E[U_1 U_2^\top]\Sigma_\pi^{-1}\E[U_2 U_1]. \end{aligned}\] Since \(\Sigma_\pi\) is positive definite, the subtracted term is nonnegative, and therefore \(\E[\varphi(O)^2] \leq \sigma_{\mathrm{naive}}^2\), with equality only when \(\E[U_1 U_2^\top] = 0\). Thus maximum likelihood estimation of a correctly specified parametric propensity-score model can reduce the asymptotic variance of the Horvitz–Thompson IPW estimator relative to using the true propensity score.
We refer to this as estimated-propensity-score variance reduction; it is sometimes called the “propensity score paradox,” though that phrase is also used for an unrelated phenomenon in matching, so the descriptive name is preferable. The phenomenon traces to Rosenbaum (1987), who observed in the context of model-based direct adjustment that weighting by an estimated propensity score can outperform weighting by the true one. It was placed on a formal semiparametric footing by Robins et al. (1992), and Hirano et al. (2003) later showed that weighting by their nonparametric series-logit propensity-score estimator attains the semiparametric efficiency bound for the ATE under the regularity conditions of that paper — a result that extends the present finite-dimensional calculation well beyond the parametric setting assumed here, though not automatically to every flexible propensity-score learner.
Scope of the result. The conclusion rests on three assumptions: (i) the propensity score model \(\pi(X;\alpha)\) is a correctly specified finite-dimensional parametric model; (ii) \(\hat\alpha\) is the maximum-likelihood estimator, so that \(U_2\) is the score and the Bartlett identity holds; and (iii) the strong-overlap and moment conditions of Section 10.9.1 are met so that \(\sigma_{\mathrm{naive}}^2\) is finite. When the parametric model is misspecified, when \(\hat\alpha\) is computed by some other method, or when \(\pi(X)\) is estimated nonparametrically by flexible machine learning, the identity \(A_{12} = -\E[U_1 U_2^\top]\) need not hold and the variance-reduction conclusion fails in this simple form. The result should not be read as the blanket claim that any estimated propensity score improves IPW variance. It is worth noting, however, that a positive result is available in the nonparametric case, by a quite different argument: Hirano et al. (2003) show that weighting by their nonparametric series-logit propensity-score estimator attains the semiparametric efficiency bound for the ATE under suitable smoothness conditions. What fails here is the particular finite-dimensional Bartlett-identity argument, not the phenomenon itself.
10.8 Variance Estimation and Confidence Intervals
Once an estimator admits the asymptotic linear representation \[\sqrt{n}(\hat\psi - \psi) = \frac{1}{\sqrt{n}}\sum_{i=1}^n \varphi(O_i) + o_p(1),\] its asymptotic variance is \(\mathrm{Var}(\hat\psi) \approx n^{-1}\mathrm{Var}\{\varphi(O)\}\). A natural estimator of this variance is obtained by replacing \(\varphi(O_i)\) with estimated influence values \(\hat\varphi_i\) and taking the sample variance: \[\hat V = \frac{1}{n(n-1)}\sum_{i=1}^n (\hat\varphi_i - \bar\varphi)^2, \qquad \bar\varphi = \frac{1}{n}\sum_{i=1}^n \hat\varphi_i.\] The asymptotically equivalent uncentered form \(\hat V = n^{-2}\sum_i \hat\varphi_i^2\) is also commonly used. Their absolute difference is \(O_p(n^{-2})\); because \(\hat V = O_p(n^{-1})\), their relative difference is \(O_p(n^{-1})\), a consequence of \(\bar\varphi = O_p(n^{-1/2})\) when \(\E\varphi(O) = 0\). We adopt the centered form throughout this book, as it has slightly better finite-sample behavior and is the version employed in Chapter 11. This yields the approximate Wald confidence interval at level \(1-a\): \[\hat\psi \pm z_{1-a/2}\,\sqrt{\hat V}.\] We use \(a\) rather than the more conventional \(\alpha\) for the significance level here and below to avoid collision with the nuisance parameter \(\alpha\) of Section 10.7.
Proposition 10.2 (Variance from the Influence Function) If \(\sqrt{n}(\hat\psi - \psi) = n^{-1/2}\sum_{i=1}^n \varphi(O_i) + o_p(1)\) with \(\E\{\varphi(O)\} = 0\) and \(\E\{\varphi(O)^2\} < \infty\), then \[\sqrt{n}(\hat\psi - \psi) \overset{d}{\longrightarrow} N\!\left(0,\, \E[\varphi(O)^2]\right).\] If, in addition, the remainder in the expansion is negligible in \(L_2\), or equivalently the sequence \(n(\hat\psi - \psi)^2\) is uniformly integrable, then \(\mathrm{Var}(\hat\psi) = n^{-1}\E[\varphi(O)^2] + o(n^{-1})\).
The distributional conclusion follows immediately from the central limit theorem applied to the i.i.d. sum. The second conclusion needs the extra hypothesis and does not follow from the first: an \(o_p(1)\) remainder controls the remainder in probability, and convergence in probability or in distribution does not imply convergence of moments. A sequence can converge in distribution to a limit with finite variance while its own variances diverge, so an integrability condition of some kind is unavoidable. With that caveat noted, the influence function plays a dual role: it characterizes both the asymptotic distribution and — under the added condition — the asymptotic variance.
10.9 Toward Efficiency: Semiparametric Models and the Efficient Influence Function
Section 10.4 showed that the influence function of an asymptotically linear estimator determines both its asymptotic distribution and its asymptotic variance. Different regular estimators of the same parameter may therefore have different large-sample variances. This raises a natural question: among all regular estimators in a given statistical model, what is the smallest asymptotic variance that no regular estimator can improve upon? Answering that question leads to semiparametric efficiency theory. In this section we introduce only the basic objects — semiparametric models, regular estimators, and the efficient influence function — as preparation for Chapter 11. The formal semiparametric machinery underlying this section is developed in Appendix C, which supplies the proofs of the claims made informally here.
10.9.1 Semiparametric Models
In causal inference, the parameter of interest \(\psi = \Psi(P)\) is a finite-dimensional quantity (e.g., the ATE), but the full distribution \(P\) of the observed data \(O = (X, T, Y)\) is never fully specified. This structure motivates the following definition.
The canonical example in this course is the ATE. Strictly speaking, the potential outcomes \(Y(0)\) and \(Y(1)\) are not part of the observed-data distribution \(P\), so the causal parameter \(\E\{Y(1)-Y(0)\}\) is not by itself a functional of \(P\). Under the identification assumptions of consistency, conditional exchangeability, and positivity (Chapter 5), however, it equals the observed-data functional \[\tau(P) = \E_P\{\mu_1(X) - \mu_0(X)\}, \qquad \mu_t(x) = \E_P(Y\mid T=t,\,X=x),\] and we study estimation of this functional in the nonparametric observed-data model \(\mathcal{P} = \{P : 0 < \pi(X) < 1\ \text{a.s.}\}\). Positivity defines the support restriction only; we restrict attention throughout to distributions \(P \in \mathcal{P}\) for which \(\tau(P)\) is finite, and at the reference distribution used for regular root-\(n\) inference we additionally require the efficient influence function to belong to \(L_2(P)\).
Here the nuisance consists of the entire joint distribution of \((X, T, Y)\) subject only to positivity; no parametric form is assumed for \(\mu_t(x)\) or \(\pi(x)\). The nuisance parameter accordingly includes the outcome regressions \(\mu_t(x)\), the propensity score \(\pi(x)\), and the marginal distribution of \(X\). These are precisely the objects that must be estimated in practice. Finite-dimensional analogues of this nuisance-estimation problem were illustrated in Section 10.7, where the nuisance was a parametric coefficient vector; Chapters 11 and 12 treat the full nuisance functions. They are not themselves the target of inference.
10.9.2 Regular Estimators
The efficiency bound is meaningful only within a restricted class of estimators. Without such a restriction, pathological estimators can behave well at a single distribution while behaving erratically under nearby perturbations, making variance comparisons meaningless from a local asymptotic point of view. The relevant class is that of regular estimators, which, informally, are estimators whose asymptotic distribution is stable under small perturbations of the data-generating distribution.
The point of indexing by a fixed \(h\) is that regularity is a statement about local uniformity: the estimator’s limiting behavior, recentered at the moving target \(\Psi(P_{h/\sqrt{n}})\), must not depend on which local direction the truth is perturbed in.
Regularity rules out pathological estimators that exploit specific features of the data-generating mechanism in a non-uniform way (so-called superefficient estimators). Under correct model specification and the regularity conditions stated for their respective statistical models, the finite-dimensional regression, IPW, and augmented estimators studied in this course are standard examples of regular estimators. The qualification matters: regularity is always relative to a specified model, and an estimator regular in one model need not be regular in a larger one. A parametric outcome-regression estimator, for instance, is not regular for the ATE in the unrestricted nonparametric model, since it fails to be consistent outside its working model. The formal theory of regular estimators and the convolution theorem underlying the efficiency bound are developed in Vaart (1998) (Chapters 8 and 25) and Tsiatis (2006) (Chapter 4).
10.9.3 The Semiparametric Efficiency Bound and the Efficient Influence Function
The central result of semiparametric efficiency theory is the convolution theorem of Bickel et al. (1993). Stating it requires the efficient influence function, so we define that object first: a function is not determined by its mean and covariance alone, and the EIF is characterized by where it sits — in the tangent space — not merely by how disperse it is.
Definition 10.4 and the ATE formula that follows from it are needed throughout Chapters 11 and 12 and should be read now. The theorem’s regularity hypotheses and its vector-valued refinement are second-reading material: on a first pass it is enough to take from Theorem 10.2 that \(V^*\) is a floor no regular estimator can beat, and that a regular asymptotically linear estimator sits exactly on that floor precisely when its influence function is \(\varphi^*\).
Definition 10.4 (Efficient Influence Function (canonical gradient)) Let \(\Psi\) be pathwise differentiable at \(P_0\) relative to \(\mathcal{P}\), with gradient set \(\mathrm{IF}(\Psi, P_0)\) and tangent space \(\mathcal{T}\). The efficient influence function (EIF), or canonical gradient, is the unique element \[\varphi^* \in \mathrm{IF}(\Psi, P_0) \cap \mathcal{T},\] obtained by projecting any gradient orthogonally onto \(\mathcal{T}\). Its covariance \(V^* = \E_{P_0}[\varphi^*(O)\varphi^*(O)^\top]\) is the semiparametric efficiency bound. Existence, uniqueness, and the variance-minimizing property are proved in Section C.6.
Theorem 10.2 (Semiparametric Convolution Theorem) Suppose \(\Psi\) is pathwise differentiable at \(P_0\) relative to \(\mathcal{P}\) with canonical gradient \(\varphi^*\) and bound \(V^* = \E_{P_0}[\varphi^*\varphi^{*\top}]\), and suppose the model satisfies the local asymptotic normality and regularity conditions required by the Hájek–Le Cam convolution theorem. Then for any regular estimator \(\hat\psi\) of \(\psi_0 \in \mathbb{R}^p\), \[\sqrt{n}(\hat\psi - \psi_0) \overset{d}{\longrightarrow} N(0, V^*) * M\] for some distribution \(M\), where \(*\) denotes convolution. Moreover, a regular asymptotically linear estimator is efficient — that is, it attains the limit \(N(0, V^*)\) with degenerate \(M\) — if and only if its influence function equals \(\varphi^*\).
The efficient influence function is not merely any valid gradient; it is the unique one lying in the tangent space, and it minimizes variance over the gradient set. Two cautions about the scope of the theorem are worth stating explicitly. First, it is not true that every regular estimator has an influence function: a regular estimator may carry a nondegenerate convolution remainder \(M\) and need not be asymptotically linear at all. What is true is that every regular asymptotically linear estimator has an influence function, and that function must be a gradient of \(\Psi\). Second, regularity alone does not imply efficiency; among regular asymptotically linear estimators, only the one whose influence function equals \(\varphi^*\) attains the bound \(V^*\) with no convolution remainder.
10.9.4 The Efficient Influence Function for the ATE
We record the EIF for the ATE as it will be derived and used extensively in Chapter 11.
10.10 Chapter Summary
This chapter introduced the statistical framework underlying estimation and inference for causal parameters.
Identification is not estimation. Identification provides a population formula, but not automatically a satisfactory estimator.
Estimating equations. Many estimators are defined as solutions to sample moment conditions \(\mathbb{P}_n\{U(O;\theta)\}=0\), with the population moment condition \(\E\{U(O;\theta_0)\}=0\) identifying \(\theta_0\) as a zero of the population analogue. Consistency of \(\hat\theta\) requires additional structure (identifiability and uniform convergence).
Asymptotic linearity. Estimating equations often yield a first-order expansion \(\sqrt{n}(\hat\theta - \theta_0) = n^{-1/2}\sum_i\varphi(O_i) + o_p(1)\), which immediately implies asymptotic normality via the CLT.
Influence functions — two distinct notions. The estimator influence function is the function \(\varphi(O)\) appearing in the expansion above, describing the first-order sensitivity of that estimator to a single observation. A gradient of a functional is a mean-zero function representing the pathwise derivative of \(\Psi\) relative to a model. They are linked in one direction: a regular asymptotically linear estimator’s influence function is necessarily a gradient.
Nuisance estimation. In causal inference, nuisance functions must be estimated, and this estimation error propagates into the target estimator. Controlling this propagation is the central statistical challenge.
Variance from the influence function. The asymptotic variance is \(n^{-1}\E[\varphi(O)^2]\), estimated by \((n(n-1))^{-1}\sum_i(\hat\varphi_i-\bar\varphi)^2\); this is consistent under the \(L_2\), moment, and cross-fitting or empirical-process conditions discussed above, and only after asymptotic linearity has been established.
Efficiency. For regular asymptotically linear estimators in a specified model, the variance of the canonical gradient, or efficient influence function, is a universal lower bound. Whether the bound is attained is a separate question: pathwise differentiability supplies the bound but does not by itself construct an estimator achieving it. An estimator is efficient when it does attain the bound, equivalently when its influence function equals the canonical gradient; for the ATE, Chapters 11 and 12 give sufficient conditions. Because the bound depends on the model, oracle estimators in restricted models may have smaller variance without contradiction. This characterization guides estimator construction in Chapter 11.
10.11 Problems
1. Estimating equations and moment conditions.
Let \(O = (X, Y)\) with \(X\) and \(Y\) scalar, \(\E(X^2) < \infty\), \(\E(Y^2) < \infty\), and \(\mathrm{Var}(X) > 0\), and define \(\theta = \mathrm{Cov}(X,Y)/\mathrm{Var}(X)\), the slope in the population simple regression of \(Y\) on \(X\). Show first that the moment condition \(\E[X(Y - \theta X)] = 0\) does not identify \(\theta\) in general: its solution is instead the regression-through-the-origin coefficient \(\theta_\circ = \E(XY)/\E(X^2)\). Writing \(\mu_X = \E(X)\) and \(\mu_Y = \E(Y)\), verify the identity \[\E\{X(Y - \theta X)\} = \mu_X(\mu_Y - \theta\mu_X),\] and conclude that \(\theta_\circ = \theta\) if and only if \(\mu_X(\mu_Y - \theta\mu_X) = 0\), that is, either \(\E(X) = 0\) or the population intercept \(\beta_0 = \mu_Y - \theta\mu_X\) vanishes. Show by example that \(\E(Y) = 0\) alone is not sufficient. (Hint: take \(\E(X) = 1\), \(\mathrm{Var}(X) = 1\), and \(Y = X - 1\).) Then give a correct estimating equation for \(\theta\) by estimating the intercept jointly, \[\E\!\begin{bmatrix} 1 \\ X \end{bmatrix}\bigl\{Y - \beta_0 - \theta X\bigr\} = 0,\] and verify that the population moment condition holds at \((\beta_0, \theta_0)\).
Let \(\theta = F^{-1}(0.5)\) be the population median, and assume \(F\) is continuous and strictly increasing in a neighborhood of its unique median. Propose an estimating function \(U(O;\theta)\) and verify the population moment condition. (Hint: consider \(U(O;\theta) = \mathbf{1}(Y \leq \theta) - 0.5\).) Explain why the continuity assumption is needed: if \(F\) has an atom at the median, \(F(\theta)\) may strictly exceed \(0.5\) and the moment condition can fail at \(\theta_0\). Note also that \(U\) is discontinuous in \(\theta\), so the smooth Taylor argument of Proposition 10.1 does not apply directly; the asymptotic linear expansion for the sample median instead follows from a Bahadur representation and requires a positive density at \(\theta_0\). Finally, observe that at the sample level \(\mathbb{P}_nU(Y;\theta) = 0\) need not have an exact solution — for instance when \(n\) is odd — so the sample median is properly defined through the empirical generalized inverse, or as an approximate Z-root satisfying \(\mathbb{P}_nU(Y;\hat\theta) = o_p(n^{-1/2})\).
Show that the OLS estimator and the IPW estimator of the ATE (Example 10.2) are both special cases of the general estimating-equation (Z-estimator) framework, in each case identifying \(U\), \(A\), and the resulting influence function.
2. Asymptotic linearity and the CLT.
Let \(\hat\psi = \bar Y\) be the sample mean of i.i.d. \(Y_1,\dots,Y_n\) with \(\E Y = \psi\) and \(\mathrm{Var}(Y) = \sigma^2 < \infty\). Write down the influence function, state its asymptotic distribution, and give a consistent estimator of \(\mathrm{Var}(\hat\psi)\).
Suppose \(\hat\psi_1\) and \(\hat\psi_2\) are two asymptotically linear estimators of the same parameter with influence functions \(\varphi_1\) and \(\varphi_2\). Show that \(\hat\psi_\lambda = \lambda\hat\psi_1 + (1-\lambda)\hat\psi_2\) is also asymptotically linear for any fixed \(\lambda \in \mathbb{R}\), and find its influence function.
Use part (b) to derive the value of \(\lambda \in \mathbb{R}\) that minimizes the asymptotic variance of \(\hat\psi_\lambda\) (allow \(\lambda\) outside \([0,1]\)). Express the answer in terms of \(\sigma_1^2 = \E[\varphi_1^2]\), \(\sigma_2^2 = \E[\varphi_2^2]\), and \(\rho = \E[\varphi_1\varphi_2]\). Your derivation should assume \(\mathrm{Var}(\varphi_1 - \varphi_2) = \sigma_1^2 + \sigma_2^2 - 2\rho > 0\); explain what happens in the degenerate case \(\mathrm{Var}(\varphi_1 - \varphi_2) = 0\), in which \(\varphi_1 = \varphi_2\) almost surely and every \(\lambda\) yields the same first-order estimator. Under what condition on \((\sigma_1^2, \sigma_2^2, \rho)\) does the optimal \(\lambda\) equal \(1/2\)?
3. Influence functions for causal estimators. Consider the ATE \(\tau = \E\{Y(1) - Y(0)\}\) identified under consistency, conditional exchangeability, and positivity. In parts (a)–(b) the nuisance functions are treated as known, so both estimators are oracle estimators in restricted models in the sense of the warning box in Section 10.6.
Assume \(\pi(X)\) and \(\mu_t(X)\) are both known. Compute the influence functions of (i) the regression estimator \(\hat\tau_{\mathrm{reg}} = \mathbb{P}_n\{\mu_1(X) - \mu_0(X)\}\); (ii) the IPW estimator \(\hat\tau_{\mathrm{IPW}} = \mathbb{P}_n\{TY/\pi(X) - (1-T)Y/[1-\pi(X)]\}\).
(A Rao–Blackwell comparison.) With \(\varphi_{\mathrm{reg}}\) and \(\varphi_{\mathrm{IPW}}\) as in Equation 10.1 and Equation 10.2, show that \[\E\{\varphi_{\mathrm{IPW}}(O) \mid X\} = \mu_1(X) - \mu_0(X) - \tau,\] using conditional exchangeability and consistency; since \(\varphi_{\mathrm{reg}}\) is a function of \(X\) alone, this says \(\E\{\varphi_{\mathrm{IPW}} \mid X\} = \varphi_{\mathrm{reg}}\). Deduce from the law of total variance that \[\mathrm{Var}(\varphi_{\mathrm{IPW}}) = \mathrm{Var}(\varphi_{\mathrm{reg}}) + \E\bigl[\mathrm{Var}\{\varphi_{\mathrm{IPW}}\mid X\}\bigr] \geq \mathrm{Var}(\varphi_{\mathrm{reg}}),\] so that in this oracle comparison the regression estimator is never asymptotically less efficient than the IPW estimator. State the condition on \(\mathrm{Var}\{\varphi_{\mathrm{IPW}}\mid X\}\) under which the two variances are equal.
The efficient influence function for the ATE is given in Equation 10.10. Verify that it has mean zero under \(P\) by computing \(\E[\varphi^*(O)]\).
4. Variance estimation.
Let \(\hat\varphi_i = \hat\mu_1(X_i) - \hat\mu_0(X_i) - \hat\tau_{\mathrm{reg}}\) be the estimated influence values for the regression estimator. Write down the plug-in variance estimator \(\hat V\) and simplify. Under what conditions is \(\hat V\) consistent for \(\mathrm{Var}(\hat\tau_{\mathrm{reg}})\)? In answering, explain why the oracle restriction matters: this \(\hat\varphi_i\) is the influence function of the estimator that treats \(\mu_t\) as known, so \(\hat V\) targets the correct variance only when the outcome regressions are effectively known, estimated on an independent and much larger sample, or otherwise contribute negligibly to the first-order expansion. Ordinary estimation of \(\hat\mu_t\) from the same sample generally adds first-order terms that this \(\hat V\) omits.
A researcher reports \(\hat\tau = 2.4\) and, from the estimated influence values, an empirical variance \(\mathbb{P}_n\hat\varphi_i^2 = 36\) based on \(n = 400\) observations, so that \(\hat V = 36/400 = 0.09\) estimates \(\mathrm{Var}(\hat\tau)\) directly. Compute a 95% Wald confidence interval. Is the treatment effect statistically distinguishable from zero at the 5% level? (Caution: \(\hat V\) is already an estimate of \(\mathrm{Var}(\hat\tau)\), not of \(\mathrm{Var}\{\varphi(O)\}\); do not divide by \(n\) a second time.)
Explain why plugging nuisance estimates into a candidate influence-function formula cannot repair an incorrect asymptotic linear expansion. In your answer, distinguish clearly between (i) the nuisance-rate conditions needed to establish that \(\hat\tau\) is asymptotically linear with influence function \(\varphi\) — which depend on the estimator and on whether its score is orthogonal — and (ii) the \(L_2\)-consistency condition \(\|\hat\varphi - \varphi\|_{L_2(P)} = o_p(1)\), together with moment and cross-fitting or empirical-process conditions, needed to estimate the variance once that expansion is already established. Explain why (ii) does not generally require \(n^{-1/2}\) nuisance rates.
5. Efficiency comparisons.
(Efficiency bounds are model-specific.) Problem 3(b) showed that, in the oracle setting where the nuisance functions are known, \(\mathrm{Var}(\varphi_{\mathrm{reg}}) \leq \mathrm{Var}(\varphi_{\mathrm{IPW}})\) always. It is also possible for \(\mathrm{Var}(\varphi_{\mathrm{reg}})\) to fall strictly below the nonparametric semiparametric efficiency bound \(\E[\varphi^*(O)^2]\). Explain why this is not a contradiction. (Hint: an efficiency bound constrains regular estimators within the model for which it was computed. Identify the model to which each of the three influence functions belongs, and explain why oracle knowledge of \(\mu_t\) corresponds to a strictly smaller model than the nonparametric one.) What, if anything, does this tell you about the relative merits of the two estimators in a practical setting where \(\mu_t\) and \(\pi\) must both be estimated?
The semiparametric efficiency bound for the ATE is \(\E[\varphi^*(O)^2]\). Show that \(\E[\varphi^*(O)^2] \leq \E[\varphi_{\mathrm{IPW}}(O)^2]\) by expanding the squared EIF. (Hint: use the law of iterated expectations to show that the cross-terms cancel.)
Give one practical reason why an efficient estimator based on the EIF may not always be preferred over a simpler, less efficient estimator.