Archive for January, 2009

How Not To Do Science (or Stats)
January 31, 2009

A few months ago, I read the Bible (of sleep and circadian research): Nathaniel Kleitman’s Sleep and Wakefulness. Though I was most intrigued by the vast differences in methodologies pre and post the 20th century, Kleitman’s commentary on “Kohlschuetter’s curve” was similarly salient:

Kohlschuetter decided that the curve depicting thresholds of arousal from acoustical stimuli throughout the progression of NREM sleep should be of normal distribution, and hence, bell-shaped. Therefore, Kohlschuetter excluded 33 of 74 (about 45 percent) observations from his analyses because these observations deviated from the normal distribution, bell-shaped curve he predicted.

Ironically enough, this curve was referred to as “Kohlschuetter’s curve,” and remained in press until a more honest statistician discovered Kohlschuetter’s malignant “mistake.” We now know, of course, that the curve of thresholds of arousal from acoustical stimuli throughout NREM is nearly linear and positive as I have graphed below:

Stage 1 sleep is characterized by light sleep (i.e imagine sleep that ensues while reading a book or gazing out the car window), stage 2 is deeper (i.e an afternoon cat nap), while stages 3 and 4 are the deepest stages of NREM sleep (i.e. remember how difficult it is to awake a 10 year old)

The CORRECT AND ACTUAL CURVE: Stage 1 sleep is characterized by light sleep (i.e imagine sleep that ensues while reading a book or gazing out the car window), stage 2 is deeper (i.e an afternoon cat nap), while stages 3 and 4 are the deepest stages of NREM sleep (i.e. remember how difficult it is to awake a 10 year old)

Moreover, can you imagine how this manner of analyzing data would transform a result section of any primary literature?!! There would be no “trends towards significance” or “approached, but did not reach significance,” or “neared significance,” no, there would just be significance (and infinite amounts of Type I errors).

HONESTY AND ACCURACY, HONESTY AND ACCURACY….