Recall that in a paired‐samples t‐test, individuals are matched on one or more characteristics. By match- ing, we reduce variability due to factor(s) we are matching on. In G*Power, we proceed as follows:
To the left, after entering all the relevant parameters (tails, effect size, significance level, power equal to 0.95, keeping the allocation ratio constant at 1, i.e.
equal sample size per group), we see that estimated sample size turns out to be n = 105 per group. Below is the power curve for an effect size of d = 0.5.
A statistical power analysis was con- ducted to estimate sample size required to detect a mean population difference between two independent populations. To detect an effect size d = 0.5 at a significance level of 0.05, at a level of 0.95 of power, a sample size of 105 per group was estimated to be required.
We can see that for the same parameters as in the independent‐samples t‐test (i.e. two‐tailed, effect size of d = 0.5, significance level of 0.05, and power of 0.95), the required total sample size is 54. Recall that for the same parameters in the independent‐samples t‐tests, we required 105 per group. This simple example demonstrates one advantage to performing matched‐pairs designs, and more gener- ally repeated‐measures models – you can achieve relatively high degrees of power for a much smaller
“price” (i.e. in terms of sample size) than in the equivalent independent‐samples situation. For more details on these types of designs, as well as more information on the concepts of blocking and nesting (of which matched samples are a special case), see Denis (2016).
G*Power can conduct a whole lot more power analyses than surveyed here in this chapter. For details and more documentation on G*Power, visit http://www.gpower.hhu.de/en.html. For more instruction and details on statistical power in general, you are encouraged to consult such classic sources as Cohen (1988).
A statistical power analysis was conducted to estimate sample size required to detect a mean pop- ulation difference using matched samples. To detect an effect size d = 0.5 at a significance level of 0.05, at a level of 0.95 of power, a total sample size of 54 subjects was estimated to be required.
In this chapter, we survey the analysis of variance procedure, usually referred to by the acronym
“ANOVA.” Recall that in the t‐test, we evaluated null hypotheses of the sort H0 : μ1 = μ2 against a statistical alternative hypothesis of the sort H1 : μ1 ≠ μ2. These independent‐samples t‐tests were comparing means on two groups. But what if we had more than two groups to compare? What if we had three or more? This is where ANOVA comes in.
In ANOVA, we will evaluate null hypotheses of the sort H0 : μ1 = μ2 = μ3 against an alternative hypothesis that somewhere in the means there is a difference (e.g. H1 : μ1 ≠ μ2 = μ3). Hence, in this regard, the ANOVA can be seen as extending the independent‐samples t‐test, or one can interpret the independent‐samples t‐test as a “special case” of the ANOVA.
Let us begin with an example to illustrate the ANOVA procedure. Recall the data on achievement from Denis (2016):
Teacher 2
1 3 4
69
70 85 95
68
67 86 94
70
65 85 89
76
75 76 94
77
76 75 93
75
73 73 91
M= 72.5
M= 71.00 M= 80.0 M= 92.67
Achievement as a Function of Teacher
Though we can see that the sample means differ depending on the teacher, the question we are interested in asking is whether such sample differences between groups are sufficient to suggest a difference of pop‑
ulation means. A statistically significant result (e.g. p < 0.05) would sug‑
gest that the null hypothesis H0 : μ1 = μ2 = μ3 = μ4 can be rejected in favor of a statistical alternative hypothesis that somewhere among the popula‑
tion means, there is a difference (however, we will not know where the differences lie until we do contrasts or post hocs, to be discussed later).
In this experiment, we are only interested in generalizing results to these specific teachers we have included in the study, and not others in the
7
Analysis of Variance: Fixed and Random Effects
population from which these levels of the independent variable were chosen. That is, if we were to theoretically do the experiment over again, we would use the same teachers, not different ones. This gives rise to what is known as the fixed effects ANOVA model (we will contrast this to the random effects ANOVA later in the chapter – this distinction between fixed vs. random will make much more sense at that time). Inferences in fixed effects ANOVA require assumptions of normality (within each level of the IV), independence, and homogeneity of variance (across levels of the IV).
We set up our data in SPSS as it appears on the left (above).