A two sample t-test is used to test whether or not the means of two populations are equal.
This type of test makes the following assumptions about the data:
1. Independence: The observations in one sample are independent of the observations in the other sample.
2. Normality: Both samples are approximately normally distributed.
3. Homogeneity of Variances: Both samples have approximately the same variance.
4. Random Sampling: Both samples were obtained using a random sampling method.
If one or more of these assumptions are violated, then the results of the two sample t-test may be unreliable or even misleading.
In this tutorial we provide an explanation of each assumption, how to determine if the assumption is met, and what to do if the assumption is violated.
Assumption 1: Independence
A two sample t-test makes the assumption that the observations in one sample are independent of the observations in the other sample.
This is a crucial assumption because if the same individuals appear in both samples then it isn’t valid to draw conclusions about the differences between the samples.
How to Check this Assumption
The easiest way to check this assumption is to verify that each observation only appears in each sample once and that the observations in each sample were collected using random sampling.
What to Do if this Assumption is Violated
If this assumption is violated, the results of the two sample t-test are completely invalid. In this scenario, it’s best to collect two new samples using a random sampling method and ensure that each individual in one sample does not belong to the other sample.
Assumption 2: Normality
A two sample t-test makes the assumption that both samples are approximately normally distributed.
This is a crucial assumption because if the samples are not normally distributed then it isn’t valid to use the p-values from the test to draw conclusions about the differences between the samples.
How to Check this Assumption
If the sample sizes are small (n
If the sample sizes are large, then it’s better to use a Q-Q plot to visually check if the data is normally distributed.
If the data points roughly fall along a straight diagonal line in a Q-Q plot, then the dataset likely follows a normal distribution.
What to Do if this Assumption is Violated
If this assumption is violated then we can perform a Mann-Whitney U test, which is considered the non-parametric equivalent to the two sample t-test and does not make the assumption that the two samples are normally distributed.
Assumption 3: Homogeneity of Variances
A two sample t-test makes the assumption that the two samples have roughly equal variances.
How to Check this Assumption
We use the following rule of thumb to determine if the variances between the two samples are equal: If the ratio of the larger variance to the smaller variance is less than 4, then we can assume the variances are approximately equal and use the two sample t-test.
For example, suppose sample 1 has a variance of 24.5 and sample 2 has a variance of 15.2. The ratio of the larger sample variance to the smaller sample variance would be calculated as:
Ratio: 24.5 / 15.2 = 1.61
Since this ratio is less than 4, we could assume that the variances between the two groups are approximately equal.
What to Do if this Assumption is Violated
If this assumption is violated then we can perform Welch’s t-test, which is a non-parametric version of the two sample t-test and does not make the assumption that the two samples have equal variances.
Assumption 4: Random Sampling
A two sample t-test makes the assumption that both samples were obtained using a random sampling method.
How to Check this Assumption
There is no formal statistical test we can use to test this assumption. Instead, we just need to make sure that both samples were obtained use a random sampling method such that each individual in the population of interest had an equal probability of being included in either sample.
What to Do if this Assumption is Violated
If this assumption is violated, then it’s unlikely that our two samples are representative of the population of interest. In this case, we can’t generalize the findings from the two sample t-test to the overall population with reliability.
In this scenario, it’s best to collect two new samples using a random sampling method.