Adventures in Signal Processing and Open Science

Month: October, 2013

Academia.edu acquires Plasmyd to bring peer review into the 21st century

Academia.edu acquires Plasmyd to bring peer review into the 21st century

I noticed this news piece today. I have previously written about open peer review platform. Most of the recent initiatives in open peer review are entirely new platforms that provide the mechanics to get open peer review going, but in my opinion a challenge for them is to attract a critical mass of users.

Academia.edu’s move seems a bit the other way around: they already have an existing science-related platform with a quite a few users, but now they are adding peer review functionality. It is not entirely clear to me whether this means open review, but the mechanism they describe could help address the challenge of how to attract sufficient numbers of qualified reviewers to such a platform. The article does hint at the possibility that academia.edu might try to “build a reveue model around their modern approach to peer review”. I am not a fan of such a model, as this is one of the things that are wrong with the traditional journal publishing model. Nevertheless, it is going to be interesting to see how it goes.

Advertisements

More on anonymity in peer review

This study lacked an appropriate control group: Two stars

I came across this post on anonymity in peer review by Jon Brock. I have previously tried to discuss pros and cons of anonymity here. I think Jon’s post is a quite good argument in favour of identifying reviewers.

In relation to this, I actually wanted to sign a review I did recently for a journal as I wanted to personally stand by my assessment of the manuscript. I asked with the editor first if this was OK with him. He explicitly requested me NOT to do so, because this was against their review policy…

Science Publishing Laboratory

Science Publishing Laboratory

Browsing on Twitter, I just stumbled on this blog by Alexander Grossmann. It looks like he has a ton of interesting reading on open scientific publishing. I found it through Giuseppe Gangarossa on Twitter.

An emerging consensus for open evaluation: 18 visions for the future of scientific publishing

An emerging consensus for open evaluation: 18 visions for the future of scientific publishing

I just found this treasure trove of papers on open evaluation in science thanks to this post by Curt Rice that sums it all up very well: Open Evaluation: 11 sure steps – and 2 maybes – towards a new approach to peer review

Third-party review platforms

I recently read this piece (http://chrissampson.me/2013/09/18/one-suggestion-for-a-new-model-of-academic-publishing/) by Chris Sampson. It is an interesting proposal and I thank Chris Sampson for writing it since it encouraged me to think so much about this possibility. I am mostly interested in the review aspect of it. Delegating the handling of peer review to third parties could for example be a good idea in terms of making the whole review process faster and more efficient – for example for the reasons argued by the team behind Libre: http://www.openscholar.org.uk/why-and-how-to-separate-scholarly-evaluation-from-academic-journals/.

I would like to weigh in with my thoughts on third-party peer review.

Funding

One important issue is that of funding such third-party “review companies”. What kind of companies are these and how do they make the money to fund their activities.

I suppose they could be non-profit organisations but could also be for-profit commercial companies. I find the former the most appealing. I guess there is nothing wrong in principle with companies making money from what they do if they do it well and provide a useful service to the scientific community, but somehow I think that such third-party review companies should exist for the sole purpose of providing competent, thorough, trust-worthy review of scientific papers and that a focus on profit might divert their attention from this goal. For example, unreasonably high profit margins are one of the reasons that large publishers such as Elsevier are currently being criticised.

However, some income is needed to run such a platform and where is that going to come from? I see several options:

  • Authors pay: this is probably not popular. It is my impression that traditional publishers are often criticised for their rather high publishing fees for (gold) open access publishing in their journals. An author fee for review could be seen as comparable to this, but I think its perception depends on how the review company is run. If it is a commercial company, an author fee might well be considered as unpopular as the mentioned OA publishing fees. If, on the other hand, the company is a non-profit able to argue that the fee simply covers the necessary (low) expenses to conduct the review process, it might be perceived as fair. Well, I am no game-theorist, sociologist, psychologist or anything relevant, so this is just my personal opinion.
  • Review company is paid by research funders: this could be a better option. In the above case of placing the fee on the authors, much of the funding for this will probably come from the researchers’ projects which are often funded by research councils etc. anyway – such as the public The Danish Council for Independent Research or The Danish Council for Strategic Research here in Denmark. It might be an idea for such research funders to simply fund the review companies directly instead, not placing the burden on individual researchers. In my opinion, this would only work fairly if the review company is a non-profit organisation. Otherwise, they would receive the same criticism for sponging off public research funding that many commercial publishers are currently subject to.
  • Review company is paid by publishers: this could be problematic. If a review company is paid for the review work by the publisher that eventually ends up publishing a particular article, the review company would be under suspicion for not being impartial. Maybe this could be less of a problem if publishers were to form a coalition that would collectively pay for all of the review being carried out. Anyway, this cost incurred on the publishers would just be carried over to publishing fees which would then be the authors’ problem.
    By the way, could impartiality also be an issue if authors pay for reviews?
  • Review company is funded by advertising on its website: I guess one could imagine this scenario although I think it is unlikely. In my opinion, this would be distasteful at best.

Chris Sampson’s post also mentions that journals would bid on papers to publish, based on the reviews. I assume he means that journals would bid as in competing for who could publish at lowest cost? Since he also suggests that all papers should be published open access, the publishers cannot charge the readers for publishing, so I assume this cost must be charged from the authors. Therefore, the competitive element must lie in which journal is able to publish at lowest cost? If the author is to pay the fee, I suppose the author also gets to choose the journal. The author might not necessarily want to select the cheapest journal; maybe a slightly higher-cost bidding journal has a better reputation and the author prefers that one.

The last part leads me to another thought: the reviews should be openly accessible as well (an issue I have discussed before on this blog). This way, there is more information available to readers for judging papers by their actual merits rather than by the journal that publishes them. Maybe this would take some of the popularity contest element out of publishing and help make publishing fees more realistic?

Publishers’ interest

I am afraid a change to the proposed model would be lobbied hard against by exisiting publishers – for at least two reasons:

  1. Review would no longer be part of the publishers’ services and so, they would have less to claim that authors need to pay for, making it more difficult for them to keep up their high profit margins.
  2. Open reviews and thereby better assessment of papers based on content rather than publication venue could work towards more equal status of publishers. This is of course something the highest-ranking journals would want to work against too.

Possible “review companies”

As I have mentioned in previous posts, several platforms have appeared recently that could take on this role of third-party reviewer. I could imagine at least: libreapp.org, peerevaluation.org, pubpeer.com, and publons.com. Pandelis Perakakis mentioned several others as well: https://thomas.arildsen.org/2013/08/01/open-review-of-scientific-literature/comment-page-1/#comment-9.

Forest Vista

seeking principles

Academic Karma

Re-engineering Peer Review

Pandelis Perakakis, PhD

Academic Website

chorasimilarity

computing with space | open notebook

PEER REVIEW WATCH

Peer-review is the gold standard of science. But an increasing number of retractions has made academics and journalists alike start questioning the peer-review process. This blog gets underneath the skin of peer-review and takes a look at the issues the process is facing today.

Short, Fat Matrices

a research blog by Dustin G. Mixon

www.rockyourpaper.org

Discover and manage research articles...

Science Publishing Laboratory

Experiments in scientific publishing

Open Access Button

Push Button. Get Research. Make Progress.

Le Petit Chercheur Illustré

Yet Another Signal Processing (and Applied Math) blog