Commenting on scientific papers

I think it is fair to say that commenting on scientific papers is broken. And with commenting I mean online comments that are publicly available, not informal discussions in journal clubs or at meetings. This definition would include discussions of papers on social media such as Twitter or Facebook. Why do I think that commenting is broken?

• the number of papers with online comments is low. For PLOS Biology we have comments on the journal platform for 11% of articles, tweets for 14% of articles and Facebook activity for 22% of articles. The numbers for Twitter and Facebook are much higher for more recently published articles, but are nowhere close to every article having at least one comment.
• even though there is a fair amount of social media activity around articles, the quality of the discussion is varied. Twitter for example seems to work mostly as an alerting service for interesting articles with little more than the title of the article in the tweet text and not much discussion.
• when comments are made, they are really hard to find coming from the article. Unless they are made on the journal platform, or the publisher tracks article-level metrics and links out to these comments.

What can be done to address these issues, i.e. increase the number of comments, increase the depth of the discussion, and make it easier to link comments to articles? Some of the thoughts that I and others have had include the following:

• lower the technical barriers for commenting by providing a common and familiar commenting platform with an attractive user interface. Many blogs (including this one) and some publishers use Disqus, which is arguably the most popular third-party commenting platform.
• develop new features that make commenting more attractive, including comments linked to specific sections of the text and notes that can be public, semi-public or private. See for example what Medium is doing, check out Hypothes.is and PeerJ Questions, or study what services such as Stackoverflow are doing.
• Link comments made in different places about the same object together, e.g. through Article-Level Metrics services.
• create incentives for scientists to comment, e.g. through Mozilla Open Badges or by making them part of a community.

On Tuesday the US National Library of Medicine launched PubMed Commons as a New Forum for Scientific Discourse:

We hope that PubMed Commons will leverage the social power of the internet to encourage constructive criticism and high quality discussions of scientific issues that will both enhance understanding and provide new avenues of collaboration within the community.

PubMed Commons is still a pilot project and in order to read or write comments you have to be a PubMed Commons participant and be signed in with your My NCBI account. PubMed Commons has some important features:

• PubMed is probably the place where most life sciences researchers search for literature. Having comments and discussion there makes perfect sense, and is probably a better place than a publisher platform that only targets particular journals. PubMed also has a reputation that is very different from social media tools that are popular, but not really familiar to most scientists.
• Access to PubMed Commons is restricted to researchers, and this is one strategy to have the comments focus on scientific discourse. It has to be seen whether the process of registering for PubMed Commons (which currently is a bit more involved than most commenting systems) is a barrier for scientists to take part in the discussion, or whether it generates an audience that makes it more likely that scientists contribute.
• For people signing in with their My NCBI account (I don’t know the percentage of PubMed users that do that on a regular basis), commenting is really easy and the interface is straightforward. The comment editor uses markdown, which makes it easy to format comments and to include links.

10/26/13: added link to the recently launched PeerJ Questions which uses a question and answer format (thanks to Jason Hoyt for reminding me).

References

Fenner, M. (2013). What Can Article-Level Metrics Do for You? PLoS Biology, 11(10), e1001687. https://doi.org/10.1371/journal.pbio.1001687