Science Pokerface

There’s no such thing as a perfect research study. Every study has its strengths and weaknesses, and as scientists-in-training, we learn that discussing the limitations of our work is an essential part of presenting our work to the scientific community. The reality, however, is that this often is seen negatively. It’s an unfortunate paradox that can make it very difficult, especially for early career researchers, to reconcile what they’ve learned is the right thing to do with what gets their work published and out there for people to read.

I recently submitted, with my graduate student, my first manuscript as a senior author to a scientific journal. It’s a simple study, with an important (but not “fatal”) limitation. As I often do in my own papers, I encouraged my student (the first author) to openly embrace this limitation in the discussion section. To explain how, despite this limitation, the study is still valuable.

I’ve had all kinds of misguided reactions from co-authors when I do this. Let the reviewers point it out, don’t actively make your own study weaker, too much “honesty” can be a bad thing.

I find it very hard to be understanding when I receive such comments. If I figured it out, most likely a reviewer, or worse yet, a reader of the future paper, will too. My main issue with this approach though is that it serves no constructive purpose.

Science isn’t a poker game – I should never feel like I have to bluff or hide something from anyone. I strongly believe that, if we can all agree that no one study is perfect, hiding a study’s imperfections in the hope that someone won’t notice is sneaky and counterproductive.

So what do you do when you receive comments from reviewers like the ones we did? One reviewer basically said “Well, the authors basically point out the flaws in their study and so I don’t see any point in it being published”. He or she offered no other constructive comments. Apparently they didn’t feel it was even worth properly reviewing for this same reason.

This says a lot about our strategy for disseminating scientific knowledge – try to get your paper out there, and if you have to hide something to do that, then so be it. That’s in and of itself problematic, but for early career researchers, this can be a huge source of confusion. You’re telling me that limitations belong in my discussion section, but I shouldn’t be too critical (whatever that means) of my own work? Where do we draw the line? What do I tell my students? My only explanation for them so far is that, regardless of how much experience some people have, they just don’t understand what science is about.

Leave a comment