F-test is a generally widespread statistical hypothesis test, based on the F-distribution. The purpose of F-test is to compare two (or more) data sets, and to draw the inference based on the test result.

The test is commonly used to decide if the variances or the means of two or more data sets are significantly different, or they differ only by chance (randomly). Our null hypothesis (H0) is that the variances (or means) of the data sets are equal, while the alternative hypothesis (Ha) is that they are different.

As in every hypothesis test, we use various mathematical formulas to generate one number (in this case F), based on which we can accept or reject the null-hypothesis, by comparing it to a critical F number (gathered from the pre-defined F table).

Sir Ronald Aylmer Fischer, an English biostatistician (combining biology and statistics), and George Waddel Snedecor, an American statistician reached new heights in the field of statistics by pioneering design of experiments, analysis of variance (ANOVA) and Snedecor’s F-distribution (or Fischer-Snedecor distribution).

In quality management (especially in quality control), we commonly use two major F-test methods:

In this section we will discuss both methods.

F-test for equality of variances is used to compare the variances (squared standard deviation) of two data sets (groups), and to make a statistical statement, if the differences of the variances are significant with a given confidence level.

Hypothesis (0): S₁² = S₂² ... variances are equal

Hypothesis (alternative): S₁² ≠ S₂² ... variances are not equal

The formula of F-test for equality of variances:

where F is the F number, S₁ is the standard deviation of group one, S₁² is the variance of group one, S₂ is the standard deviation of group two, S₂² is the variance of group two.

Example: we want to compare if the variance of group one is statistically different from group two with 95% confidence. We use the F-test for this decision.

Step 1: calculate the value of F value.

The groups are featured by the following descriptive statistics:

F is calculated by the following formula: F = S₁² / S₂² = 64.0 / 51.84 = 1.2346

So our F value for the two groups gives 1.2346, which we need to compare to a critical F value to see if we can accept or reject the null hypothesis. We will use the F tables for this. We have set up a needed confidence level at the beginning of the test, which is 95%. This means our alpha level (α) is 5%.

As we make a two-tailed test, our alpha will be half of the 5%, so 0.025 (in case of two-tailed tests, we have to divide alpha by two). This is very important, as there are different F-tables for different alpha levels. It is also important that:

Step 2: define the degrees of freedom values.

The degrees of freedom (df) is equal to the sample size of the group minus 1 (df = n - 1). The group one’s df is 12, while group two’s df is 10. In the F tables the columns represent the numerator degrees of freedom, while the rows represent the denominator degrees of freedom. Our df (numerator) is 12 and our df (denominator) is 10, so it is easy to check it in the F-table, which belongs to the α = 0.025 alpha level.

IMPORTANT! Such as with the variances, in case of a left-tailed test, you have to use the df number of the group with smaller variance in the numerator, and the other df number in the denominator. Once you have your Critical F value, you have to reciprocate it, and then use the received number to compare it with your calculated F value.

Step 3: use the df values and the alpha level for selecting the critical F value in the F table.

The table gives us the critical F statistic value, which is 3.6209. Now we have to compare it to the calculated F value. If the calculated F value exceeds the critical F value, we can reject the null-hypothesis, which would mean that the means are significantly different at the 95% confidence level.

Step 4: conclusion

Conclusion: the calculated F value does not exceed the critical F (1.2346 < 3.6209), so we have to accept the null hypothesis. From quality standpoint, we see no significant difference between the two populations of product / process characteristics at 5% significance level (which is equal to 95% confidence level).

Remark: confidence level (1- α) = 95%, significance level (α) = 5%.

The one-way ANOVA F-test is a more complex test compared to equality of variances, and it is used for the comparison of means. By doing an ANOVA F-test we can compare even more than two groups, and check if their means are statistically equal at a given confidence level.

First we set up our null-hypothesis and alternative hypothesis.

Hypothesis (0): µ₁ = µ₂ = µ_k ... means are equal

Hypothesis (alternative): µ₁ ≠ µ₂ ≠ µ_k ... means are not all equal

In case of one-way ANOVA F-test, the F value represents the ratio of explained variance vs unexplained variance, or in other words the between-group variance vs the within-group variance.

F = "between–group variance" / "within-group variance"

Example: we want to compare if the means of three groups (data sets) are statistically equal, or different from each other with 95% confidence. We use the one-way ANOVA F-test for this decision.

The values in the groups are the following:

Step 1: calculate the overall mean ( x̄ ), also called as the grand mean.

= (4 + 6 + 5 + 8 + 10 + 9 + 7 + 7 + 7) / 9 = 7

Step 2: calculate the group means (x̄₁; x̄₂; x̄₃).

x̄₁ = (4 + 6 + 5) / 3 = 5

x̄₂ = (8 + 10 + 9) / 3 = 9

x̄₃ = (7 + 7 + 7) / 3 = 7

Step 3: calculate the total sum of squares.

SST = (4 – 7)² + (6 – 7)² + (5 – 7)² + (8 – 7)² + (10 – 7)² + (9 – 7)² + (7 – 7)² + (7 – 7)² + (7 – 7)² =

= 9 + 1 + 4 + 1 + 9 + 4 + 0 + 0 + 0 = 28

Step 4: calculate the total degrees of freedom by multiplying the number of groups by the number of samples in one group, and subtract 1 from the number.

DF_Total = m * n -1 = 3 * 3 -1 = 9 -1 = 8

Step 5: calculate the sum of squares within-group (SSW), by getting the sum of squares for all the 9 values.

SSW = (-1)² + (1)² + (0)² + (-1)² + (1)² + (0)² + (0)² + (0)² + (0)² = 1 + 1 + 0 + 1 + 1 + 0 + 0 + 0 + 0 = 4

Step 6: calculate the within-group degrees of freedom (DF_Within).

DF_Within = m (n – 1) = 3 * (3 – 1) = 3 * 2 = 6

Step 7: calculate the between-group sum of squares (SSB).

SSB = n (group1 mean – grand mean)² + n (group2 mean – grand mean)² + n (group3 mean – grand mean)²

SSB = 3 * (5 – 7)² + 3 * (9 – 7)² + 3 * (7 – 7)² = 3 * (-2)² + 3 * (2)² + 3 * (0)² = 3 * 4 + 3 * 4 + 3 * 0 = 12 + 12 = 24

Step 8: calculate the between-group degrees of freedom (DF_Between).

DF_Between = m -1 = 3 – 1 = 2

Step 9: calculate the between-group mean square (MSB) and the within-group mean squares (MSW).

MSB = SSB / DF_Between = 24 / 2 = 12

MSW = SSW / DF_Within = 4 / 6 = 0.6667

Step 10: summarize all data in a comprehensive table:

Filling up the table with the calculated numbers:

Step 11: calculate the F value, be dividing MSB by MSW.

F = MSB / MSW = 12 / 0.6667 = 18.0

The table is useful to make a crosscheck, as SSB + SSW must be equal to SST. Also DF_Between + DF_Within must be equal to DF_Total.

From the numbers, we can already see, that the total variation is mainly coming from the variation between the groups. This may give us an indication, that the F value will be high, and we already see a number of 18.0.

But do we know yet if we can reject the null-hypothesis? Well, not, as we have to finish our test by getting the critical F value, and to compare it with our calculated F value (also called F-statistic).

The degrees of freedom for the numerator (between-group) is 2, and the degrees of freedom for the denominator (within-group) is 6. Using the α = 0.05 F-table (having the confidence level of 95%), with the given DF values, we see, that our critical F value is 5.1433.

Step 12: conclusion

As our calculated F value exceeds (much higher) the critical F (gained from the table), we can reject the null-hypothesis at the 95% confidence level. In practice, it means, that we can state the group means significantly differ from each other.

Using F-tests in practice can support decision making, as we can statistically consider the difference between independent populations. When you compare different lots / shifts, etc. in your production based on a given product or process characteristic, you can support your decision with statistics, making an F-test. It is mainly useful, when the samples are large (over 100 pcs / sample).

When conducting a two-tail test, you have to divide alpha by 2, so in case of a two-tailed test and 95% confidence, you will need to use the 0.025 F-table instead of using the 0.05 F-table.

In case of F-test of equal variances the larger variance must be placed in the numerator, while the smaller in the denominator.

F-test is quite sensitive to normality, so the underlying population from which the samples were taken should be normally distributed. This sensitiveness abates with larger sample sizes, and the sample size of each group should be near to each other (large and equal sample sizes are the best for an F-test). In addition, F-test gives better indication if the standard deviation of the samples do not differ greatly (according to Sullivan: highest S should be less, than double the lowest S, however other statisticians are more cautious).

When there are only two sample means to compare, the F-test is basically the same, as the two sampled T-test. In such a case the F-statistic is equal to the square of the T value (F = t²).

To easily conclude statistical tests (such as F-test or T-test), use statistical software, which calculates the statistics for you in milliseconds. It is also advisable to use such test for Statistical Process Control (SPC), and not just performing range, mean, C_p, C_pk calculations.