Generalized p-value

To describe the idea of generalized p-values in a simple example, consider a situation of sampling from a normal population with the mean ${\displaystyle \mu }$, and the variance ${\displaystyle \sigma ^{2}}$. Let ${\displaystyle {\overline {X}}}$ and ${\displaystyle S^{2}}$ be the sample mean and the sample variance. Inferences on all unknown parameters can be based on the distributional results

${\displaystyle Z={\sqrt {n}}({\overline {X}}-\mu )/\sigma \sim N(0,1)}$

and

${\displaystyle U=nS^{2}/\sigma ^{2}\sim \chi _{n-1}^{2}.}$

Now suppose we need to test the coefficient of variation, ${\displaystyle \rho =\mu /\sigma }$. While the problem is not trivial with conventional p-values, the task can be easily accomplished based on the generalized test variable

${\displaystyle R={\frac {{\overline {x}}S}{s\sigma }}-{\frac {{\overline {X}}-\mu }{\sigma }}={\frac {\overline {x}}{s}}{\frac {\sqrt {U}}{\sqrt {n}}}~-~{\frac {Z}{\sqrt {n}}},}$

where ${\displaystyle {\overline {x}}}$ is the observed value of ${\displaystyle {\overline {X}}}$ and ${\displaystyle s}$ is the observed value of ${\displaystyle S}$. Note that the distribution of ${\displaystyle R}$ and its observed value are both free of nuisance parameters. Therefore, a test of a hypothesis with a one-sided alternative such as ${\displaystyle H_{A}:\rho <\rho _{0}}$ can be based on the generalized p-value ${\displaystyle p=Pr(R\geq \rho _{0})}$, a quantity that can be easily evaluated via Monte Carlo simulation or using the non-central t-distribution.

• Gamage J, Mathew T, and Weerahandi S. (2013). Generalized prediction intervals for BLUPs in mixed models, Journal of Multivariate Analysis}, 220, 226-233.
• Hamada, M., and Weerahandi, S. (2000). Measurement System Assessment via Generalized Inference. Journal of Quality Technology, 32, 241-253.
• Krishnamoorthy, K. and Tian, L. (2007), “Inferences on the ratio of means of two inverse Gaussian distributions: the generalized variable approach”, Journal of Statistical Planning and Inferences, Volume 138, Issue 7, 1, Pages 2082-2089.
• Li, X., Wang J., Liang H. (2011). Comparison of several means: a fiducial based approach. Computational Statistics and Data Analysis, 55, 1993-2002.
• Mathew, T. and Webb, D. W. (2005). Generalized p-values and confidence intervals for variance components: Applications to Army test and evaluation, Technometrics, 47, 312-322.
• Wu, J. and Hamada, M. S. (2009) Experiments: Planning, Analysis, and Optimization. Wiley, Hoboken, New Jersey.
• Zhou, L., and Mathew, T. (1994). Some Tests for Variance Components Using Generalized p-Values, Technometrics, 36, 394-421.
• Tian, L. and Wu, Jianrong (2006) “Inferences on the Common Mean of Several Log-normal Populations: The Generalized Variable Approach”, Biometrical Journal.
• Tsui, K. and Weerahandi, S. (1989): "Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters". Journal of the American Statistical Association, 84, 602–607
• Weerahandi, S. (1995) Exact Statistical Methods for Data Analysis Springer-Verlag, New York. ISBN 978-0-387-40621-3

• XPro, Free software package for exact parametric statistics