A question about Effect Sizes arose at NTENRED, so I thought I’d summarise what the Effect Size is, how one is used and criticisms of the use of the Effect Size in educational research.
For an in depth exploration of all of this, I recommend reading the excellent critique of Effect Sizes provided by blogger who writes as Ollie Orange 2 (http://ollieorange2.wordpress.com).
What is the Effect Size?
Briefly, the Effect Size is the difference between two means divided by the standard deviation. For non-mathematicians, the standard deviation is a ‘measure of spread’, which gives you an idea of how far from the mean a typical observation might be. See my post here for a bit more detail on this, or have a look at something like this.
So the Effect Size is a way of checking how big the difference between in two means actually is, taking into account how typical the mean is of the data which it summarises.
It’s important to realise that the Effect Size is prefaced by a definite article – it’s a particular thing. It’s not an indefinite thing – although you often encounter ‘an effect size’ in an educational context, this isn’t how Effect Sizes were developed.
How is the Effect Size used
The short answer is ‘incorrectly’, in an educational context. The Effect Size should be used to check whether an experiment will have enough data from which to draw valid conclusions before the experiment takes place.
In most educational research, the Effect Size is used to compare different methods of teaching or outcomes of a change in educational process. There is no justification for this.
Criticisms of the use of Effect Sizes in educational research
As noted above, http://ollieorange2.wordpress.com is a good place to start. Further criticism comes here: http://www.learningspy.co.uk/myths/things-know-effect-sizes/. If you know of any others, please post in the comments.
For an in depth exploration of all of this, I recommend reading the excellent critique of Effect Sizes provided by blogger who writes as Ollie Orange 2 (http://ollieorange2.wordpress.com).
What is the Effect Size?
Briefly, the Effect Size is the difference between two means divided by the standard deviation. For non-mathematicians, the standard deviation is a ‘measure of spread’, which gives you an idea of how far from the mean a typical observation might be. See my post here for a bit more detail on this, or have a look at something like this.
So the Effect Size is a way of checking how big the difference between in two means actually is, taking into account how typical the mean is of the data which it summarises.
It’s important to realise that the Effect Size is prefaced by a definite article – it’s a particular thing. It’s not an indefinite thing – although you often encounter ‘an effect size’ in an educational context, this isn’t how Effect Sizes were developed.
How is the Effect Size used
The short answer is ‘incorrectly’, in an educational context. The Effect Size should be used to check whether an experiment will have enough data from which to draw valid conclusions before the experiment takes place.
In most educational research, the Effect Size is used to compare different methods of teaching or outcomes of a change in educational process. There is no justification for this.
Criticisms of the use of Effect Sizes in educational research
As noted above, http://ollieorange2.wordpress.com is a good place to start. Further criticism comes here: http://www.learningspy.co.uk/myths/things-know-effect-sizes/. If you know of any others, please post in the comments.