How to respond to a review of a scientific paper

[Prologue: We decided last year to talk about peer review in our lab meeting, as a training exercise for the students. First we covered how to review, then how to respond to a review (this subject of this post). In general, I think we should actively teach these things. Most of us learn by watching how our advisors do it, and it works for some of us....but why not make this standard graduate student education? 

As a coauthor and  editor and reviewer, I often see poor response letters, leading me to believe that many people are just not being given good instruction in how to craft a response. The lab meeting discussion topic led me to write my version of how to respond to a review. It's fairly consistent with some treatments like this, and this, but personalized to reflect what one guy does who has done this a bunch of times. I don't claim this is the one true way, just a way that has worked out for me.

When we discussed this, there were other faculty present who added their personal tips as well, which showed that we (the professors) mostly followed the same principles, but differed a little bit in our approach.  That was good for students to see. These kind of instructional lessons are best paired with real examples. We also spent some time looking at a couple of my real response letters. I can imagine turning this into an exercise, giving a student a comment and asking them to respond to it.]


How to respond to a review of a scientific paper:

Part 1. Choose your own adventure 

1. (Accept) minor revisions – Celebrate. Tell your coauthors so they can too. Get on the minor edits within a few days, while the editor still remembers the paper …it may speed things up.

2. Revise & resubmit – Read comments, share outcome with authors, alert them that you may be requesting input soon. If it’s easy, you can start revising as soon as you can.
If it’s not so easy, chances are you’ll be feeling sort of defensive about some of the comments. Put it away until next week. Don’t respond with a “hot take”.

3. Reject with invitation to resubmit – Read comments, share outcome with authors, tell them whether you are feeling like resubmitting or not and get their take. The decision to resubmit or not will depend on whether you are willing/able to rebut or apply all of the comments.  Usually, it is better to resubmit to that journal if willing/able to address the comments (you’ll stand a somewhat better chance there than another equal journal). 
These days, this is the new “revise & resubmit” so journals can talk about how exclusive they are. That’s my cynical take, anyway. So they might actually want your paper after all.
Again, put it away until next week. Watch a you tube video of “Hitler and reviewer 3”. It helps. 

4. Reject – Inform your coauthors.
Skim the review, don’t dwell on it.

After a few days, read the comments in detail.

Very rarely you may challenge the decision. The journal will have a formal process for this. To challenge you must be able to argue convincingly and succinctly that the reviewers were flat-out and demonstrably wrong, careless or biased and this led to a wrong decision by the editor. The subject/handling editor is the one who usually makes the decision. Absolutely do not make a habit of this, it’s kind of a nuclear option.

Assuming you are not challenging….
With a calm mind, determine whether there is good advice in the reviews. There almost always is. Even though you’re not bound to apply these comments, it is smart to seriously consider them since they may help armor you in future reviews. It is not unlikely that one of the same reviewers will review again for the other journal, and they will be annoyed if you ignored the hours they volunteered to do your review.

Part 2. So, you are revising for a resubmission to the same journal (adventures 1-3 above)

You need to produce a revised manuscript and a response letter to the reviewers.

1. Check journal instructions. They may want a revision where changes are marked somehow, in addition to a clean version. Know what they want before doing anything. Also check what the deadline is.

2. To make your response letter, copy all comments into a doc verbatim. This will become your point by point response. Always draft in a word processing file, so you can save the letter for your records.

3. (Optional) Annotate your manuscript with comments for you. It can be useful to copy the major comments, into comment bubbles into a version of your ms in the place where you think it should addressed. This is just for your own reference.

4. Work through the comments, making the needed change in the ms. If the journal wants your changes marked in some way, e.g. colored font or track changes, do that. In the appropriate spot in your response letter (after the corresponding comment) either (a) describe what you did to address that comment, or (b) explain why you didn’t make the requested change. Make it easy to distinguish response from the comment, e.g. I like to use bold italics for my responses. (Though, note that depending on the manuscript submission system you may have enter the info into a box online that does not allow special formatting….check first) 

5. (Recommended) Do the easy stuff first. Leave the comments you need to think about until later sessions. It will probably be easier psychologically for you that way, plus it gives you time to think through the tough ones.

6. It’s the corresponding author’s job to do handle the response, so if that’s you, take the lead. But it is also common to ask coauthors for input or thoughts on specific comments that they might be more qualified to tackle, e.g. the person who did the super specialized spatial model in your paper might want to handle questions about that. If you are a student, definitely solicit your advisor’s advice on your responses.

7. Add an opener to your response. Put a short paragraph on the front of your letter, in front of the detailed responses to comments. It’s basically a “thank you” note for everyone’s volunteer time to improve your work, and acknowledgement that the paper is better (hopefully you feel this way, otherwise maybe you caved a little too much).

Dear Dr. Famous –
We wish to thank you and the reviewer for your efforts. {you might point out something specific like: especially the statistical advice and the many thought-provoking suggestions for the discussion}. We have considered all of the constructive suggestions, and applied (all/the majority). As a result, we feel this draft is much stronger. Below you will find your comments and those of the reviewers reproduced verbatim, followed by our responses in bold italics (or whatever).

8. Offer your coauthors a chance to comment on your response before resubmitting. You’re communicating on their behalf, ethics dictate they should be offered a chance to see what you are resumbmitting & give a thumbs up. It’s not impolite to limit time because you usually have a deadline. 

Part 3. When responding, should you “fight” or bend the knee? Or compromise?

Usually your response is going to be a mixture of giving the reviewer what they want (and actually improves or at least does not harm the paper), saying “no thanks” on some comments, and meeting them partway on others.

Here’s the key things: 
YOU DO NOT HAVE TO DO EVERYTHING A REVIEWER SAYS, IF YOU DON’T AGREE. 
YOU ABSOLUTELY DO HAVE TO CONSIDER EACH COMMENT, & DECIDE IF THE REVIEWER MAY BE FULLY OR PARTIALLY RIGHT.

IF THEY ARE NOT RIGHT, YOU MUST CLEARLY ARGUE YOUR CASE (TO THE EDITOR AND THAT REVIEWER).

IF THEY ARE RIGHT, YOU SHOULD BE CHANGING SOMETHING.

NEVER JUST IGNORE A COMMENT. YOU MUST HAVE SOME RESPONSE TO ALL. IF YOU DON’T YOU ARE TAKING A MAJOR RISK.

I'M USING ALL CAPS BECAUSE YOU REALLY SHOULD NOTICE THIS PART! I'M TYPE SHOUTING!!!!

1. Review all comments with a calm mind. Little known fact: reviewers are actually right pretty often! If 2 reviewers comment on the same thing, that’s a strong indication that there’s really something there to work on. When you pretty much agree with the reviewer, apply their suggestion. SO EASY! 

Also, go ahead and give them the minor stuff, even if you’re only kind of on board. They usually tell you exactly how to make them content.

If you can, bend the knee on most matters. Guess what?! Chances are you’re now at least halfway (numerically) through the comments.

2. When should you fight? It gets tricky when you don’t really agree with a comment….should you “fight” (mostly NOT do what the reviewer suggests and explain why in a rebuttal), or meet them halfway, or cave? 

As in life, fighting should not be your first option, nor your default. A smart person chooses their fights. They fight when it matters, and when they at least have a chance of winning. 
There’s 3 factors really: A. How strongly do you feel that you are right and the reviewer is wrong? B. Are you going to be able to clearly and effectively articulate your case without emotion to the decider (editor), and maybe even convince the reviewer? C. Are you fighting a lot already? Can you engage one more fight without appearing closed to constructive criticism (i.e. looking like an obstinate shit)?

3.  How should you fight? There are many shades of “fighting”, some overt, some subtle. It should be polite and respectful by default. Yes, you can fight politely. If someone else is rude, respond with emotionless language, do not escalate. Most instances of fighting actually result in meeting somewhere in the middle, i.e. you have fought to defend some of your ground. Here’s a few hypotheticals:

Scenario 1: I’m right, here’s why
Problem: The reviewers don’t seem to get why you used model selection. They think more people would understand a simple full-factorial ANOVA, than your overcomplicated fancy-pants showboating. You feel strongly that not only have you thought through the options, but you have valid reasons for your approach.

Solution: First try to see through the eyes of the reviewer. Did you express, in a way that a typical reader could understand, why you made the choice you did? If no, you clarify the wording in the manuscript. If yes take note of where this was clearly expressed. Then compose your clear-headed argument in support of your approach, with the intent to persuade. 

Then in your response letter you might apologize that you were unclear in the presentation of your reasoning, explain your change, and offer a little more lengthy explanation of why you are right. “We believe we failed to communicate our reasoning for selecting this approach, and that led to these comments. We apologize for the confusion. Now, the text states: “We applied model selection because we needed to identify only the most crucial experimental factors that should be carried forward into experiment two.” The model selection approach is optimized for this purpose. We do in fact include the full factorial model in our model set (see supplemental table 2), and found that it is not among the best models”

If you were clear from the beginning and you think the reviewers blew it, then skip the mea culpa, state your argument first, then tack on something like: “We feel this was explained succinctly on L334-336. If we have not been clear, we would be grateful for specific suggestions to improve clarity.”

Outcome: The reviewers and editor will see you actually considered the comment, that you are motivated by reason and you mean to do a good job (check!). Hopefully they will agree that your reasoning is sound (check!). If it works all the way, that’s a WIN. You keep the analysis 100% the way you think is right, yay!

Scenario 2: Divergent reviewers.
Problem: You have a section in your discussion you are really proud of. The writing is good, and it provides an interesting, unconventional interpretation of the data and some hypotheses for future researchers to test. Reviewer 1 finds it overly subjective and recommends removal to reduce the length of the paper. Reviewer 2 had some positive comments about it and is intrigued to follow up in their future research. You can’t make them both happy at the same time. 

Solution: With regard to the subjectivity argument, you can explain to Reviewer 1 that you feel the discussion is the main place in the manuscript where the author can interpret the data and that this is important to do. Show them that your language makes it clear that you are building a hypothesis to be tested rather than a firm statement of fact. You remind them that proposing new interpretations is one way that science may advance in new directions. As evidence, you point out that reviewer 2 did not level the same criticism and finds value in the new interpretation. You tell reviewer 1 that you understand their concerns about the length of the section, and as a result you were able to reduce the text by 20%.

Outcome: You show you listened to both reviewers, even though you agree more with one than the other. WIN! You keep 80% of the section that you love.

Scenario 3: Switcheroo a.k.a. I can’t give you that, how about this?
Problem: Reviewer 3 has a hard time believing the conclusions of your experiment. They think that in your diversity-function study, your design makes it impossible to distinguish distinct mechanistic explanations: complementarity, and the sampling effect. They argue that you really need a different experiment that is able to partition these two mechanisms, or you should stop focusing your discussion on the complementarity mechanism. You have no time or money for another experiment, so this comment appears to be a stake in your heart. You cannot deliver what they want, exactly.

Solution: You investigate some numerical methods of partitioning these two effects, and you apply them to the data you have. You have to add this to your methods, and results and make a figure. While you’re at it, you go back to your last 4 experiments with similar diversity gradients and do the same thing, allowing you to plot (in general) the degree to which diversity affects function via complementarity. You add that to your methods and results. You present it as kind of a mini-metanalysis, that enhances the evidence for the generality of your findings. 

Outcome: You could not deliver what the reviewer wanted, but you offered something totally different instead. With luck, both reviewer and editor will be satisfied with your effort. WIN! You’re not dead on arrival, you kept your experiment, you kept your complementarity-focused discussion (100% of your content), and with some extra work, you enhanced the interpretability of the results (bonus 20%).

4. As you gain experience you’ll find your comfort level in your choice to give in or fight on individual comments. You may become more or less fighty, and probably your fighting will be more subtle….like a stealthy ninja. 

Part 4. Ok, but what do you actually say in response to a comment, and how do you say it?

1. For comments that are simply grammar, spelling, word choice edits and the like, just say: Done. or Corrected. 
(i.e., No discussion is needed…we did it)

2. For more substantive comments in which you take the reviewer advice and addressed the comment, actually explain what you did. If its not obvious where the change is, you can either tell the editor/reviewer where the changed part now resides (e.g. L133), or just copy into your rebuttal the altered sentence (e.g. We have modified our statement to reflect the results of this study, and removed overgeneralizations. The sentence now reads: ________).

3. If you’re not going to apply the comment (i.e. you’re engaging in some form of “fight”), you must state a reason. Tone and clarity are key here, because you must convince the editor and the reviewer that you seriously considered the matter and have a reasonable argument for not going the suggested direction. 

You very well might have to pull out extra evidence like citations. Once in a while you might have to do an analysis or prep another figure to support your argument. This is uncommon, but it is a good test of your resolve in this fight.

4. Never, ever, ever insult someone. For example, don’t say “That is a rather disingenuous argument”. 

Instead say, “We strongly disagree with this argument for the following reasons:…..” or if you want to adopt a softer tone : “We understand your concerns, but respectfully disagree for the following reasons:…..”

5. Once in a while, a reviewer catches a key error, or suggests a great idea that you didn’t have before. 
Do: Briefly acknowledge such instances. 
For example: Great catch! That was a typo, and it led us to check the entire appendix again which is now error free.
Thanks! That was an excellent idea, and we agree that it improved the flow of the discussion substantially. 

Don’t: Thank the reviewers for every. single. comment. It reeks of the kind of slobbery deference to authority that is the antithesis of science. A blanket “thanks for the review statement”, and occasional acknowledgement of the most useful comments will suffice, and show that you appreciate this process. Hopefully you do, despite it's quirks.

6. If you did something really dumb, like put the same table in twice as 2 different tables (yeah, I did that), it is appropriate to briefly acknowledge the error and apologize for the confusion it caused. 

When you realize that your presentation errors led to confusion, go ahead and acknowledge that. 

For example: “Both reviewers appear confused by this section, indicating that we did not communicate our point effectively. We apologize for the confusion.”

Don’t: Apologize constantly. Same reason: slobbery deference to authority is the antithesis of science. 

7. This process is peer review. If you remember this simple fact, it will help you get your tone right. Always remember that and address the reviewers and editors respectfully and professionally, but as peers, i.e. equals. You are neither the inferior or the superior of the reviewers & editors. Communicate to them as you would like your professional peers to communicate to you. Just as they should respect the work you put into the manuscript, you should respect the volunteer time they are putting into reviewing.