Chi-Square Distribution

This is a distribution made for analyzing how taken samples from a $Z$ distribution (Normal Distribution) is distributed. So we take $Z_i$ values from the distribution and use them to measure how much they vary from the expected.

This is used to see how confident a sample variance to the population variance.

Since $Z$ values can be $+$ or $-$, we use squared sums to avoid cancellations.

Interactive - https://www.desmos.com/calculator/wygsiv0nlc

Core Idea

if $X$ follows a chi-square distribution,

$$X \sim \chi^2(k)$$

• $k$ - Degree of Freedom

Statistic Definition

if $Z_1, Z_2, \dots, Z_k$ are independent standard normal random variables where $Z_i \sim \mathcal{N}(0, 1)$, the sum of their squares follows a chi-square distribution with $k$ degrees of freedom.

$$Q = \sum_{i=1}^{k} Z_i^2 \sim \chi^2(k)$$

For example if we take 10 samples from a normal distribution for one point in the Chi-Square distribution, that has the degree of freedom of $10$. Furthermore, that means each plot point in the Chi-Square is calculated by getting 10 random samples from the normal distribution. (This is the theoretical approach)

$$Z^2 = \frac{(X - \mu)^2}{\sigma^2} = \frac{(\text{Observed} - \text{Expected})^2}{\text{Variance}}$$

PDF

$$f(x; k) = \frac{1}{2^{k/2} \Gamma\left(\frac{k}{2}\right)}\cdot x^{(k/2) - 1} e^{-x/2}$$

• $k$ - No. of DOF

Properties

Mean (Expected Value)

$$E[X] = k$$

• Expected value is equal to the degree of freedom.

Variance

$$V[X] = 2\cdot k$$

• Variance ($\sigma^2$) is twice the degree of freedom.

Maximum

• Has a maximum value when DOF is 2 or more.

Chi-Square becomes a normal distribution when DOF increases.

Cumulative Probability

In practical tables, the distribution is given as cumulative distribution. Also the side it measures is important (left-tailed/right-tailed). If right tailed, shaded area is probability of $\alpha$. If left tailed, shaded area is probability of $1-\alpha$.

Practical use with Sample Variance

$$\chi^2 = \frac{(n - 1) s^2}{\sigma^2}$$

• $s$ - Sample variance
• $\sigma$ - hypothesized population variance
• $n$ - sample size

This is useful since in real world we almost never know the true population variance or it is costly to find.

$$s^2 = \frac{\sum (x_i - \bar{x})^2}{n - 1}$$

So to cancel it out and get the theoretical formula for the Chi-Square distribution, we use $n-1$ in the numerator.

$$(n - 1) s^2 = (n - 1) \left[ \frac{\sum_{i=1}^{n} (X_i - \bar{X})^2}{n - 1} \right] = \sum_{i=1}^{n} (X_i - \bar{X})^2$$

Substituting this back into the formula yields:

$$\frac{(n - 1) s^2}{\sigma^2} = \sum_{i=1}^{n} \left(\frac{X_i - \bar{X}}{\sigma}\right)^2$$