Two-step M-estimator

The class of two-step M-estimators includes Heckman's sample selection estimator,[1] weighted non-linear least squares, and ordinary least squares with generated regressors.[2]

To fix ideas, let ${\displaystyle \{W_{i}\}_{i=1}^{n}\subseteq R^{d}}$ be an i.i.d. sample. ${\displaystyle \Theta }$ and ${\displaystyle \Gamma }$ are subsets of Euclidean spaces ${\displaystyle R^{p}}$ and ${\displaystyle R^{q}}$, respectively. Given a function ${\displaystyle m(;;;):R^{d}\times \Theta \times \Gamma \rightarrow R}$ , two-step M-estimator ${\displaystyle {\hat {\theta }}}$ is defined as:

${\displaystyle {\hat {\theta }}:=\arg \max _{\theta \in \Theta }{\frac {1}{n}}\sum _{i}m{\bigl (}W_{i},\theta ,{\hat {\gamma }}{\bigr )}}$

where ${\displaystyle {\hat {\gamma }}}$ is an M-estimate of a nuisance parameter that needs to be calculated in the first step.

Consistency of two-step M-estimators can be verified by checking consistency conditions for usual M-estimators, although some modification might be necessary. In practice, the important condition to check is the identification condition.[2] If ${\displaystyle {\hat {\gamma }}\rightarrow \gamma ^{*},}$ where ${\displaystyle \gamma ^{*}}$ is a non-random vector, then the identification condition is that ${\displaystyle E[m(W_{1},\theta ,\gamma ^{*})]}$ has a unique maximizer over ${\displaystyle \Theta }$.

When the first step is a maximum likelihood estimator, under some assumptions, two-step M-estimator is more asymptotically efficient (i.e. has smaller asymptotic variance) than M-estimator with known first-step parameter. Consistency and asymptotic normality of the estimator follows from the general result on two-step M-estimators.[4]

Let {V_i,W_i,Z_i}ⁿ
_i=1 be a random sample and the second-step M-estimator ${\displaystyle {\widehat {\theta }}}$ is the following:

${\displaystyle {\widehat {\theta }}:={\underset {\theta \in \Theta }{\operatorname {arg\max } }}\sum _{i=1}^{n}m(v_{i},w_{i},z_{i}:\theta ,{\widehat {\gamma }})}$

where ${\displaystyle {\widehat {\gamma }}}$ is the parameter estimated by maximum likelihood in the first step. For the MLE,

${\displaystyle {\widehat {\gamma }}:={\underset {\gamma \in \Gamma }{\operatorname {arg\max } }}\sum _{i=1}^{n}\log f(v_{it}:z_{i},\gamma )}$

where f is the conditional density of V given Z. Now, suppose that given Z, V is conditionally independent of W. This is called the conditional independence assumption or selection on observables.[4][5] Intuitively, this condition means that Z is a good predictor of V so that once conditioned on Z, V has no systematic dependence on W. Under the conditional independence assumption, the asymptotic variance of the two-step estimator is:

${\displaystyle \mathrm {E} [\nabla _{\theta }s(\theta _{0},\gamma _{0})]^{-1}\mathrm {E} [g(\theta _{0},\gamma _{0})g(\theta _{0},\gamma _{0})^{\mathrm {T} }]\mathrm {E} [\nabla _{\theta }s(\theta _{0},\gamma _{0})]^{-1}}$

where

${\displaystyle {\begin{aligned}g(\theta ,\gamma )&:=s(\theta ,\gamma )-\mathrm {E} [s(\theta ,\gamma )\nabla _{\gamma }d(\gamma )^{\mathrm {T} }]\mathrm {E} [\nabla _{\gamma }d(\gamma )\nabla _{\gamma }d(\gamma )^{\mathrm {T} }]^{-1}d(\gamma )\\s(\theta ,\gamma )&:=\nabla _{\theta }m(V,W,Z:\theta ,\gamma )\\d(\gamma )&:=\nabla _{\gamma }\log f(V:Z,\gamma )\end{aligned}}}$

and ∇ represents partial derivative with respect to a row vector. In the case where γ₀ is known, the asymptotic variance is

${\displaystyle \mathrm {E} [\nabla _{\theta }s(\theta _{0},\gamma _{0})]^{-1}\mathrm {E} [s(\theta _{0},\gamma _{0})s(\theta _{0},\gamma _{0})^{\mathrm {T} }]\mathrm {E} [\nabla _{\theta }s(\theta _{0},\gamma _{0})]^{-1}}$

and therefore, unless ${\displaystyle \mathrm {E} [s(\theta ,\gamma )\nabla _{\gamma }d(\gamma )^{\mathrm {T} }]=0}$, the two-step M-estimator is more efficient than the usual M-estimator. This fact suggests that even when γ₀ is known a priori, there is an efficiency gain by estimating γ by MLE. An application of this result can be found, for example, in treatment effect estimation.[4]

• Generated regressor
• Heckman correction
• Feasible generalized least squares
• Two-step feasible generalized method of moments

• Adaptive estimator