Peer review or peanut gallery?

The flagellum paper saga continues, most recently with an interesting post and especially the ensuing discussion at T. taxus (also linked to from PT). Unfortunately, many people are still focusing on whether the Liu and Ochman article is scientifically valid, which is not something any scientist is going to take someone’s word for. We need to see the arguments formally presented and reviewed by other experts. Maybe that sounds too restrictive — well, you can’t have it both ways. To be frank, I find the notion that people consider this spectacle an example of the self-correcting nature of science (e.g., 1, 2) very alarming. It suggests either an only passing familiarity with the actual peer review system, or a willingness to let it slide while simultaneously holding it up as a primary argument against anti-evolutionists.

Here is Matzke’s explanation for his actions:

As one additional consideration, add the fact that the L&O paper was clearly aimed to get public attention, media attention, and to serve as a confident rebuttal to ID. It had the authority (PNAS) and rhetoric (We have debunked ID! Never mind that it has already been debunked!) to go far. I found out about the paper from a science journalist looking for an assessment, and it was already getting picked up on the blogs (and, we later found out, Science magazine). So I only had two real choices: (a) stay quiet and let a large number of pro-evolution people shoot themselves in the foot by triumphantly citing this paper as the latest and greatest anti-ID publication, only to have it collapse (as it would inevitably) later on, to the embarrassment of everyone and providing a permanent talking point to the ID guys about how evolutionists will uncritically accept any old thing that supports their position, or (b) do what I did and be frank about it. As I expressed in my initial blogpost, I didn’t like doing what I did, but it wasn’t really much of a choice.

Reality check. I saw a grand total of three blog posts about the paper from pro-evolutionists. Mine, which gave equal time to Nick’s model. Dennehy’s, which, like mine, was tentative in its acceptance of the paper, and Harrison’s, which also mentions Matzke’s model and mostly just summarizes the paper. The first two bloggers are professional scientists, the third is a freshman student. There was also a story in Science, which I agree was not particularly well done. Readers of this blog will be aware of my complaints about scientific reporting and the all too common trend toward sensationalism at the expense of accuracy. That’s about it. None of the blog posts claimed that this study refuted ID (whatever it means to refute an untestable idea), they merely described it as further evidence that an evolutionary approach is applicable to the question of flagellar origins, and that genome data will prove useful in this endeavor.

On the anti-evolution side, I saw a similar number of posts. These said little about the science (no surprise), and focused mainly on Matzke’s rhetoric. I see nothing to be embarrassed about in the three evolutionists’ blog posts above, and lots to be concerned about in terms of arming the anti-evolutionists with sound bites that excuse them from even having to read the paper under discussion. I therefore reject the rationale of needing to protect evolutionists from shooting themselves in the foot. As to whether the Liu and Ochman paper was intended to be an anti-ID PR piece as Matzke has suggested, I invite everyone to read the article. It is open access, as are about half a dozen other papers in every issue of the journal. From my reading, their focus is entirely on the science, and any comments about the evolution of complex structures being subject to confusion or speculation can be interpreted in a variety of ways, including as a statement about previous evolutionary discussion. Then read Pallen and Matzke (2006) for comparison (the latter is a nice paper, but note its explicitly stated message).

I am finding it hard to be any clearer about this. I have no problem with the paper being refuted, if that is what happens. But if so, it will occur through peer review, not by trying to convince non-experts online. Let me also point out that it is not a problem that Matzke is not a practicing scientist (I referenced his model very favorably in my post, and I respect his expertise in this area) nor that the discussion took place on blogs. My big problem, as I have stated again and again, relates to the way this was handled by Matzke, other bloggers, and blog commenters. More specifically, I am referring to 1) Matzke’s unnecessary polemics, 2) the immediate and unquestioning acceptance of Matzke’s claims by commenters and other bloggers without having read the paper or having been presented with any detailed analysis (a trend which, one would hope, is usually only observed among anti-evolutionists), 3) the misleading impression that this is even remotely similar to how peer review operates (namely, make a conclusion first, then gradually add more evidence as necessary to convince non-experts), and 4) letting fear of anti-evolutionism drive the way science is discussed.

I believe Matzke owes Renyi Liu and Howard Ochman a public apology, regardless of whether the paper was bunk or not. Until then, I maintain the hypothesis that this episode has alienated practicing scientists and has rendered them less likely to engage on blogs or to consider this a reasonable medium for serious scholarly discussion. So far, n = 1 for sure.

Update: I am rescinding my call for a public apology. I think it would be a good move, but I can’t argue that it is required. I have been in discussion with Nick and others and I think that in light of the context it makes the most sense to let Nick communicate with Liu and Ochman (or not) as colleagues in whatever way he thinks appropriate. I should also like to clarify that it is not just Nick’s approach that could alienate scientists, but the environment that it engendered.

On the flipside, I am still hopeful that blogs could be a venue for scholarly discussion, and I would like to see it appeal to and be accessible by non-experts while preserving a strong sense of academic rigor.