Presentation introducing a better hypothesis testing methodology: Effect Size.

1. Beyond p value: Effect Size April 4, 2008 Guy Lion

4. Effect Size Measures Cohen’s d = (Mean Pilot – Mean Control )/Pooled Stand. Deviation Cohen’s d is similar to the unpaired t test t value. It relies on Standard Deviations instead of Standard Errors. Hedges’ g is a more accurate version of Cohen’s d Hedges’ Cohen’s d Adjustment for small sample

5. Pilots vs Controls In this example, there is a 1.9 Standard deviation difference between Pilots and Controls.

6. Effect Size Info

7. An Example

8. ES Confidence Interval The Effect Size standard deviation formula allows to build Confidence Intervals around Effect Size values.

9. 1 st Nonparametric test: Gamma Index Recalculated Gamma Index to make it the same sign as Cohen's d and Hedges' g.

10. 2 nd Nonparametric test: Cliff Delta Cliff Delta ranges between +1 when all values of one group are higher than the values of the other group and – 1 when reverse is true. Two overlapping distributions would have a Cliff Delta of 0.

11. Cliff Delta efficiently… [(1 x 0) – (1 x 8)]/10 = -0.8

12. Cliff Delta vs Cohen’s d If distributions are Normal, Cliff Delta = Percent of Nonoverlap within a Cohen's d framework.

13. Effect Size and Sample Size requirement The above formula results from the algebraic transformation of: t stat or Z value = Difference in Means/Group Standard Error.

14. Testing Sample Size