More rambling about science

1.  (No) Fear

With all my senseless ranting about ESP, I forgot to mention another psychology paper making the rounds through the popular press.  Like the ESP paper, the authors and journal are both prominent within the field of psychology.  Unlike the ESP paper this paper, The Human Amygdala and the Induction and Experience of Fear, has already been published.

The paper describes a single subject who has a specific genetic condition that resulted in a lesion of her amygdala.  Subsequently, the authors describe how she can no longer experience fear.  The account is interesting and not without scientific merit, but the authors (and the popular press) have, in my mind, overstepped the data.  For one thing, the subject has damage to a wide variety of structures of surrounding the amaygdala.  Also the paper doesn't describe experimental data, but rather a series of interview responses and anecdotes.  I don't mean to imply that this sort of data is invalid, but more that they don't support the strong conclusions made in the paper and in the popular press.

2.  Strong Claims in Science

I mentioned in the ESP post that I think this sort of thing is a big problem in psychology, and indeed I think its a systemic problem that affects every branch of science.  Publications (and citations) are the primary currency in academics, with careers literally decided on the number (and quality) of published work.   However, publication does not occur in a vacuum.  Papers that make very strong claims (ESP exists!) will be more heavily cited, more broadly discussed, and thus more valuable as academic currency than papers that make relatively weaker claims.  The problem is that the latter papers often include better scientific methods than the former.

This all depends on how you define proper scientific methodology (and science itself) of course, but it is becoming increasingly clear to me that the way to become an eminent scientist is to make strong claims that may (or may not) be based on evidence rather than applying scrupulous methods and getting into the nitty gritty aspects of research.  I haven't done any formal work on this, but I can't even count the number of papers I've read in top tier journals that feature shoddy methods. 

3.  So what?

There really isn't another metric for judging an academic other than publications and citations, so I don't think the situation is going to change.  However, I think the increasing prevalence of science blogs and other forums for discussion may do a lot for revealing the flaws in scientific papers.  Perhaps this will lead researchers to be more cautious about making strong claims and applying good methodology but probably not.