On this page we will cover the F-test, which is used to test whether the population variance is equal between two populations.

This test differs from the two-sample z-test and two-sample t-test in that the test statistic isn’t the difference between sample statistics, but the ratio of the sample variances. To run this test, we will introduce a new distribution (F-distribution) and introduce its attributes.

The F-test is also the basis of Analysis of Variance that we will cover later.

TL:DR

The F-test is used to test the null hypothesis that two populations have the same variance
$LaTeX: H_{0}:\sigma_{1}^{2} = \sigma_{2}^{2}$
$LaTeX: H_{A}: \sigma_{1}^{2}\ne\sigma_{2}^{2}$
If we have two different populations with sample variances and and if
- both populations are normally distributed
- population variances $LaTeX: \sigma_{1}^{2}$ and $LaTeX: \sigma_{1}^{2}$ are equal
Then the sampling distribution of the ratio of sample variances, denoted by F, follows an F distribution
The F-distribution is a family of curves (similar to the t-distribution) where the distribution shape depends on
- Numerator degrees of freedom ( $LaTeX: df_{N}$ = sample size of observations in the sample placed in the numerator of F minus 1)
- Denominator degrees of freedom ( $LaTeX: df_{D}$ = sample size of observations in the sample placed in the denominator of F minus 1)
The total area under an F-distribution curve is equal to 1, and the mean of the F-distribution is approximately equal to 1 for any/all $LaTeX: df_{N}$ , $LaTeX: df_{D}$
Evaluating an F-test is like all other hypothesis tests – calculate a standardized test statistic from the observed data and evaluate it by calculating critical values and/or p-values from a reference distribution (F)

Key terms

F-distribution
Variance
Degrees of freedom

Applications of the F-test

Up to this point, we have focused on tests of proportions (z-test) and means (t-test), but we might also want to know whether one population is more variable than another. For example, we might ask

Are heights of men (population 1) more variable than heights of women (population 2), irrespective of differences in mean height
Is annual rainfall more variable in Seattle (population 1) or Portland (population 2)

To address these questions, we need to develop a new testing approach geared towards testing variability, rather than means or proportions. The F-test serves this purpose, but also acts as the basis for Analysis of Variance, which is a test for comparing multiple mean values.

Basis of the F-test

The F-test is used to test the null hypothesis that two populations have the same variance ( $LaTeX: \sigma^{2}$ ) based on sample variances ( $LaTeX: s^{2}$ ).

The associated null and alternative hypotheses are that

$LaTeX: H_{0}$ : $LaTeX: \sigma_{1}^{2} = \sigma_{2}^{2}$

$LaTeX: H_{A}$ : $LaTeX: \sigma_{1}^{2} \ne\sigma_{2}^{2}$

To run this test we require

Samples of size $LaTeX: n_{1}$ and $LaTeX: n_{2}$ are randomly selected from two potentially different populations
Samples are independent, and representative of the populations being assessed
The populations for each sample are normally distributed

If these are met, and the null hypothesis of equal variance is true, then the ratio of sample variances, denoted by F

$LaTeX: F=\frac{s_1^2}{s_2^2}$

Is distributed according to an F distribution. The F-statistic above serves as our standardized test-statistic for this procedure, and has the following properties

If the null hypothesis is true ( $LaTeX: H_{0}$ : $LaTeX: \sigma_{1}^{2} = \sigma_{2}^{2}$ ) then the expected value for F is 1
If sample 1 is more variable than sample 2, then F will be greater than 1 (extending to infinity if $LaTeX: s_{2} ^{2}$ is very small)
If sample 2 is more variable than sample 1, then F will be less than 1 (extending to 0 if $LaTeX: s_{1}^{1}$ is very small, or $LaTeX: s_{2} ^{2}$ is very large)

Therefore, possible test-statistics range from 0 to infinity. Furthermore, if our sample sizes were small and/or uneven, we might expect the observed ratio to differ more from the null expectation of 1, and for the sampling distribution to be asymmetric.

F-distribution

The F-distribution is a family of continuous probability density curves, like the normal distribution and t-distributions we have covered so far.

All F-distributions are positively skewed and are defined for positive values (0 to infinity) only, which makes it a suitable distribution for the F test-statistic. Like all probability density functions, the area under the curve is equal to 1.

However, unlike the t-distribution or standard normal distribution that we have used thus far, the mean value of the F-distribution is approximately equal to one.

The F-distribution depends on two parameters: the numerator degrees of freedom ( $LaTeX: df_{N}$ ) and the denominator degrees of freedom ( $LaTeX: df_{D}$ ). Collectively these two parameters alter the shape of the resulting F-distribution.

F-distributions illustrations

Image description

The F-distribution is always right-skewed, but the shape is determined by df_N and df_D that are linked to the sample sizes of the data being compared. The images above show different versions of the F-distribution as determined by the degrees of freedom parameters.

For the F-test, we define that

$LaTeX: df_{N} = n_{N} - 1$
$LaTeX: df_{D} = n_{D} - 1$

Where $LaTeX: n_{N}$ and $LaTeX: n_{D}$ are the sample size of observations in the numerator and denominator samples, respectively. This means that the shape of the F-distribution depends on both the overall sample size, as well as whether the sample sizes were uneven.

The resulting F-distribution serves as our sampling distribution, from which we calculate critical regions and/or p-values to test our null hypothesis.

However, because the F-distribution is asymmetric, the calculation of p-values and critical regions will depend on which of the two sample variances is larger.

If the F-statistic is less than 1 ( $LaTeX: s_{1}^{2}$ is smaller than $LaTeX: s_{2}^{2}$ ) then critical values, rejection regions, and p-values are calculated from the left-tail of the F distribution.

If the F-statistic is greater than 1 ( $LaTeX: s_{1}^{2}$ is larger than $LaTeX: s_{2}^{2}$ ) then critical values, rejection regions, and p-values are calculated from the right-tail of the F distribution.

Numerator Variance Small and Larger illustrations

Image description

Long description here

The reason for doubling the area under the curve is that this is still a two-tailed test (we reject the null of no difference if one sample variance is either much higher, or much lower than the other).

Example application and calculation steps

To demonstrate a two-sample F-test of variance we will evaluate the following scenario.

Two models of car are being compared regarding their fuel economy (miles per gallon of fuel). For each model, a sample of 25 cars are driven around the same test loop, and their fuel economy is recorded.

Model 1

n = 25, mean = 27 mpg, sd = 1.1 mpg

Model 2

n = 25, mean = 24 mpg, sd = 1.8 mpg

We want to test whether the variance in fuel economy differs between these two models at a significance level α = 0.05.

We will be using the same Prepare – Check – Calculate – Conclude steps.

Prepare

List hypotheses and identify the population parameter(s) of interest, the direction of the test (left, right, two-tailed), the significance level, α, sample statistics and sample size

The population parameters are the variances in fuel economy for each make of car, which we will label $LaTeX: σ_1^{2}$ and $LaTeX: σ_{2}^{2}$

We are testing whether the variance is the same between these two populations. Therefore, the corresponding null and alternative hypotheses are

$LaTeX: H_{0}:\sigma_{1}^{2} = \sigma_{2}^{2}$

$LaTeX: H_{0}: \sigma_{1}^{2}\ne\sigma_{2}^{2}$

Our sample statistics are the sample sizes, and sample variances

$LaTeX: n_{1}$ = 25, $LaTeX: s_{1}^{2}$ = 1.12 = 1.21

$LaTeX: n_{2}$ = 25, $LaTeX: s_{2}^{2}$ = 1.82 = 3.24

The test statistic is the ratio of these two variances

$LaTeX: F=\frac{s_1^2}{s_2^2}=\frac{1.21}{3.24}=0.373$

Because the numerator is smaller than the denominator we evaluate this by looking at the left-side of the F-distribution, using the significance level α = 0.05 to calculate critical values and/or p-values.

Check

Verify conditions of the sampling distribution under the null hypothesis.

The requirements for the F-test are that the data consist of a random and representative sample of the populations being assessed, and that the data are normally distributed.

To assess this we would plot a histogram of the data to visually assess normality.

car mpg graphs

Image description

To assess whether the populations are normally distributed we can and should always consult the data we have. If our sample data approximately follow a normal distribution, without any extreme values, then we may safely assume that the populations are normally distributed. In the above example, we use histograms to perform this check on each of the sample datasets.

Based on this assessment we have no reason to doubt the assumption of normality.

Calculate

if the conditions hold, compute the standard error, the standardized test statistic, and identify the rejection region and critical values, OR calculate the p-value of the observed statistic

For the F-test there is no standard error calculations, as the F-statistic is already standardized relative to the F-distribution.

So, the only thing we need to do is identify the appropriate F-distribution, the critical values and rejection regions and the p-value.

The F-distribution depends on the numerator and denominator degrees of freedom, which are each equal to the sample size minus 1

$LaTeX: df_{N} = n_{N}$ – 1 = 25 – 1 = 24

$LaTeX: df_{D} = n_{D}$ – 1 = 25 – 1 = 24

As the numerator variance is smaller we assess statistical significance either by

Calculating the critical value as the α/2 quantile of the F-distribution. The rejection region is F-values less than this

Calculating the p-value as twice the area under the F-distribution to the left of the observed F-value

Numerator Variance Smaller graph

Image description

the F-distribution is the null distribution for a test of two population variances. To evaluate an F-test we can either calculate critical values for a given significance level using the qf() function, or calculate the p-value based on the observed F test-statistic using the pf() function.

Conclude

Evaluate the hypothesis test by comparing the test statistic to the rejection region (or p-value relative to α) and provide a conclusion within the context of the problem.

Because the F-statistic falls within the rejection region (or p-value is less than the significance level), we reject the null hypothesis that $LaTeX: H_{0}: \sigma _{1}^{2} = \sigma_{2}^{2}$ at α = 0.05 (F = 0.373, $LaTeX: df_{N}$ = 24, $LaTeX: df_{D}$ = 24), in favor of the hypothesis that the population variances are different. From this we would conclude that one of the car models has more variable fuel efficiency than the other.