Measures of Success

It's with split interest that I note the publication of Kuhl (2013) and the dissenting comments from two of the paper's reviewers, Jon Brock and Dorothy Bishop. As Brock and Bishop rightly note, Kuhl (2013) resorts to an after-the-fact cherry picking of data from a broad array of dubious measures, and presents those post-selected findings as significant. The dissents of Brock and Bishop are part of a slowly growing movement against such questionable techniques — a few in the scientific community have begun to recognize (perhaps with some egg on their faces) that all this post-hoc data mining might not be the best route forward in the advancement of human understanding. I applaud this growing dissent, feeble though it may be.

But there's also a bitter irony to be found here. As noted in my previous post, it is Dorothy Bishop herself who has pronounced, without the slightest hint of disingenuousness, the recipe for success in today's science: “If you want to make your way in the scientific world, there are two important things you have to do: get grant funding and publish papers.” Well, let's compare Kuhl (2013) against those criteria, shall we? Let's see, the paper was supported by grants from the National Institute of Mental Health and the National Institute of Child Health and Human Development. So get grant funding, check. And of course we're all discussing this paper precisely because it has appeared in the highly regarded PloS ONE journal. So publish papers, check. Heck, with the aid of a tacked-on co-author, Kuhl (2013) has even managed to score some media exposure, which will no doubt lead to further grant funding and more publications. So, bonus check check. By Dorothy Bishop's criteria for modern scientific success, Kuhl (2013) could only be described as a stunning achievement!

Listen, I know that Kuhl, Brock, Bishop and all their scientific colleagues mean well, but I'm one of those old-fashioned folks who tends to judge people on what they do, not on what they mean or say. And the one thing I can say Kuhl, Brock, Bishop and all their scientific colleagues manage to do in common is stand firmly behind, indeed even form, the machinery of modern science (grant funding, formal publication, peer review, academic credentialing, co-authorships, etc.). So I'm having a hard time seeing how any of them have earned the right to complain about the costs of that machinery. Because make no mistake about it, one of the costs of that machinery is the massive proliferation of papers such as Kuhl (2013). It's as inevitable as 2 following 1.

I'm impressed when the dissenters are willing to speak out against the problem, but I'll be even more impressed when the dissenters quit justifying and forming the conditions of the problem.




Kuhl PK, Coffey-Corina S, Padden D, Munson J, Estes A, et al. (2013) Brain Responses to Words in 2-Year-Olds with Autism Predict Developmental Outcomes at Age 6. PLoS ONE 8(5): e64967. doi:10.1371/journal.pone.0064967