Quadratic form (statistics)

From Wikipedia, the free encyclopedia

If $ε$ is a vector of $n$ random variables, and $Λ$ is an $n$ -dimensional symmetric square matrix, then the scalar quantity $ε'Λε$ is known as a quadratic form in $ε$ .

1 Expectation
2 Variance
- 2.1 Computing the variance in the non-symmetric case
3 Examples of quadratic forms
4 See also

[edit] Expectation

It can be shown that

$\operatorname{E}\left[\epsilon'\Lambda\epsilon\right]=\operatorname{tr}\left[\Lambda \Sigma\right] + \mu'\Lambda\mu$

where $μ$ and $Σ$ are the expected value and variance-covariance matrix of $ε$ , respectively, and tr denotes the trace of a matrix. This result only depends on the existence of $μ$ and $Σ$ ; in particular, normality of $ε$ is not required.

[edit] Variance

In general, the variance of a quadratic form depends greatly on the distribution of $ε$ . However, if $ε$ does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that $Λ$ is a symmetric matrix. Then,

$\operatorname{var}\left[\epsilon'\Lambda\epsilon\right]=2\operatorname{tr}\left[\Lambda \Sigma\Lambda \Sigma\right] + 4\mu'\Lambda\Sigma\Lambda\mu$

In fact, this can be generalized to find the covariance between two quadratic forms on the same $ε$ (once again, $Λ 1$ and $Λ 2$ must both be symmetric):

$\operatorname{cov}\left[\epsilon'\Lambda_1\epsilon,\epsilon'\Lambda_2\epsilon\right]=2\operatorname{tr}\left[\Lambda _1\Sigma\Lambda_2 \Sigma\right] + 4\mu'\Lambda_1\Sigma\Lambda_2\mu$

[edit] Computing the variance in the non-symmetric case

Some texts incorrectly state the above variance or covariance results without enforcing $Λ$ to be symmetric. The case for general $Λ$ can be derived by noting that

ε'Λ'ε = ε'Λε

$\epsilon'\Lambda\epsilon=\epsilon'\left(\Lambda+\Lambda'\right)\epsilon/2$

But this is a quadratic form in the symmetric matrix $\tilde{\Lambda}=\left(\Lambda+\Lambda'\right)/2$ , so the mean and variance expressions are the same, provided $Λ$ is replaced by $\tilde{\Lambda}$ therein.

[edit] Examples of quadratic forms

In the setting where one has a set of observations $y$ and an operator matrix $H$ , then the residual sum of squares can be written as a quadratic form in $y$ :

$\textrm{RSS}=y'\left(I-H\right)'\left(I-H\right)y$

For procedures where the matrix $H$ is symmetric and idempotent, and the errors are Gaussian with covariance matrix $σ 2 I$ , $RSS / σ 2$ has a chi-square distribution with $k$ degrees of freedom and noncentrality parameter $λ$ , where

$k=\operatorname{tr}\left[\left(I-H\right)'\left(I-H\right)\right]$

$\lambda=\mu'\left(I-H\right)'\left(I-H\right)\mu/2$

may be found by matching the first two central moments of a noncentral chi-square random variable to the expressions given in the first two sections. If $H y$ estimates $μ$ with no bias, then the noncentrality $λ$ is zero and $RSS / σ 2$ follows a central chi-square distribution.