How Not To Do Science (or Stats)

A few months ago, I read the Bible (of sleep and circadian research): Nathaniel Kleitman’s Sleep and Wakefulness. Though I was most intrigued by the vast differences in methodologies pre and post the 20th century, Kleitman’s commentary on “Kohlschuetter’s curve” was similarly salient:

Kohlschuetter decided that the curve depicting thresholds of arousal from acoustical stimuli throughout the progression of NREM sleep should be of normal distribution, and hence, bell-shaped. Therefore, Kohlschuetter excluded 33 of 74 (about 45 percent) observations from his analyses because these observations deviated from the normal distribution, bell-shaped curve he predicted.

Ironically enough, this curve was referred to as “Kohlschuetter’s curve,” and remained in press until a more honest statistician discovered Kohlschuetter’s malignant “mistake.” We now know, of course, that the curve of thresholds of arousal from acoustical stimuli throughout NREM is nearly linear and positive as I have graphed below:

Stage 1 sleep is characterized by light sleep (i.e imagine sleep that ensues while reading a book or gazing out the car window), stage 2 is deeper (i.e an afternoon cat nap), while stages 3 and 4 are the deepest stages of NREM sleep (i.e. remember how difficult it is to awake a 10 year old)

The CORRECT AND ACTUAL CURVE: Stage 1 sleep is characterized by light sleep (i.e imagine sleep that ensues while reading a book or gazing out the car window), stage 2 is deeper (i.e an afternoon cat nap), while stages 3 and 4 are the deepest stages of NREM sleep (i.e. remember how difficult it is to awake a 10 year old)

Moreover, can you imagine how this manner of analyzing data would transform a result section of any primary literature?!! There would be no “trends towards significance” or “approached, but did not reach significance,” or “neared significance,” no, there would just be significance (and infinite amounts of Type I errors).

HONESTY AND ACCURACY, HONESTY AND ACCURACY….

Advertisements

One Response

  1. Hi, this is a comment.
    To delete a comment, just log in, and view the posts’ comments, there you will have the option to edit or delete them.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: