How Reviewers should Gain Credit

I have discussed the issue of peer-review before, and my frustration at being asked to review for journals with high submissions fees (n.d.a) something for which we as academics gain very little credit for.

I recently fell across by an economist who acts as an editor for a journal (n.d.b) which has an interesting perspective. One comment he makes rings true.

When you receive a request to review, please respond, and please respond as quickly as possible. I am always baffled by how many people fail to ever respond to requests to review.

— Marc F Bellemare

I have always felt slightly guilty about this, although I do get a fairly large quantity of spam about reviewing, much of which is not particuarly directed at me and often with short deadlines. And much of which says “please log into our terrible system to accept or decline”. Which you need a password to do. Which you have to request from a different part of the terrible system. But, ultimately, I am aware that in many cases, there is a person at the end, sending a perfectly reasonable request. And I do not always reply. For me, this goes against the grain, as I try to be helpful and courteous whenever I can, as life is more pleasant this way. There are limits, of course; this is my job not my hobby, and if I gain little value from an activity, at some point the time cost is going to become unjustifiable.

Recently, there has been some interest in Publons, a new website which attempts to change this. With Publons, you send your review in and then later your send in the confirmation email from the journal about your review, and this is associated with your profile. It opens up the peer-review process because the reviews end up public, making a process that I have attempted to do for myself before (n.d.c) considerably easier. For that purpose, Publons seems like a good thing. But, also, it attempts to bring some credit to the process; over time, you build your profile up and the metrics associated with it.

Now, all of this seems like a nice idea. Reviewers gain credit. But, there again, what do these metrics mean? And how important are they to me, yet another metric from yet another social network? It’s nice to get a new Twitter follower, a facebook like or even “earn” some Eve-Online ISK. But, to justify the effort I need something more than nice.

Marc Bellemare offers a different perspective, that of an hard-working journal editor, during which he describes his feelings about the responses to requests for reviews.

Consider these quotes:

In this business, “I’m too busy” is a really bad excuse and often means “I don’t care.” Have you ever met a pre-tenure academic who was not busy? […​] if you truly care about something, you will make time for it, even if it means getting up at 4 am to do it

— Marc F Bellemare

From my background, I like the idea that you should do reviewing because you care about it, because you care about the science (or economics), and you want to do a good job for yourself and your community. There again, if you have to get up at 4am to do pretty much anything, then yes, you are indeed too busy. There might be a few exceptions, I guess, mostly associated with travel which sometimes requires odd hours (I wrote some of this at 4am, and yes, it was travel), but I do not and will not work at 4am routinely. Perhaps, this makes me unusual. Maybe many of my papers have been reviewed by an exhausted colleague at 4am; it would explain the grumpy or just plain confused tone of some of them.

Interestingly, though, this rule appears not to apply to everyone.

That said, it is perfectly legitimate to decline a request to referee if you are senior in your field […​] But if you are a recently minted PhD and you tell me “I just don’t have time for this right now,” […​] chances are you are not going to impress anyone.

— Marc F Bellemare

Personally, I do not understand this at all. If you are a “senior” in your field, then you probably have a permanent position, it’s probably reasonably well-paid, and probably reasonable secure. If you are a “recently minted PhD”, you probably have none of these things, in addition to a significant debt that you are servicing. It’s even less clear to me why, under such circumstances, you would want to get up at 4am, however much you care. Surely, he has a stronger argument than this for why we should review? , the only other answer he gives is this:

[…​] the journal’s editors will often be among the people asked to provide external letters for tenure and promotion cases, and good citizenship in your discipline tends to get rewarded.

— Marc F Bellemare

Frankly, this seems a pretty weak argument — good citizenship tends to get rewarded; just a tendency and you have to wait till the tenure committee stage of your career if you haven’t left before then? I also dislike the implicit threat (don’t review, and we’ll rubbish your tenure case), which explains why he excludes seniors, as I dislike the general tone of arrogance and entitlement in the whole article.

Putting that aside, the more important question is how much do tenure and promotion committees really care? In my experience, they are far more interested in high scoring papers for REF or that your research is sufficiently expensive (n.d.d/) The world has become metric-based (n.d.e/) and good citizenship is a marginal issue at best.

I was entertained, therefore, to read that Nature is experimenting with actually paying reviewers for their reviews (n.d.f) As the publishers say: “Authors submitting a manuscript to Scientific Reports can choose a fast-track peer-review service at an additional cost.” The costs are passed directly onto the authors of course. The practical upshot is that “Authors who opt-in to fast-track will receive an editorial decision (accept, reject or revise) with peer-review comments within three weeks”

Of course, the general feedback to this annoucement was not particularly positive (n.d.g/) and it seems that Scientific Reports will not be carrying on with this process (n.d.h) Concerns were expressed that it could affect the independence of review, that the reviewers would be no good (presumably because they are poor “newly-minted” PhDs), and that it would favour scientists with lots of money.

But I am less convinced that it is such a bad thing. The reviewers are not paid by the scientists directly; it’s unclear that the current system produces good reviews and the entire system favours scientists with lots of money; after all, they pay the oft-mentioned newly-minted PhDs to do the work and write their papers anyway.

Ultimately, I have come to the conclusion that the contribution of the reviewers need to be acknowledged in some way; the general way that we acknowledge work is money. The journals get paid, so why should the reviewers not do so also? And unlike publons metrics or unquantified brownie points from editors, I can spend money at the supermarket.

So, I think it is time that reviewing was put onto a reasonable footing, and the same footing that most of the rest of our lives run on. Don’t get me wrong here, I do enjoy my work: fundamentally, research retains its excitement for me, and teaching is a privilege. But I would probably do very little of either if I won the lottery and had enough to live on without. It’s my job, and I do it because I get paid.

I have, therefore, put my terms and conditions for reviewing up for everyone to see. I think that they are reasonable and proportionate. Of course, I have not worried about my reviews impressing anyone for years, and I hope that I will plan my workload so that I never have to get up at 4am. More likely, of course, the journals will just say no (one has already), and I will get asked to review less. But, either way, I will be able to respond as quickly as possible and that will remove the one piece of guilt that I feel about the whole process.

Comments welcome as always.