Reviewing a scientific paper means reading it critically, evaluating whether the methods and conclusions hold up, and writing a structured report that helps the editor decide on publication and helps the authors improve their work. A thorough review typically takes several hours spread across multiple readings, and the process follows a predictable sequence from first read-through to final recommendation.
How Peer Review Works
The process starts when a journal editor invites you to review a submitted manuscript. If you accept, you’ll get access to the manuscript files and sometimes the authors’ cover letter, figures, tables, and other metadata. Your job is to read the paper carefully, assess its scientific quality, and submit a written report with your comments and a recommendation. The editor collects reviews from multiple reviewers, weighs them alongside their own reading, and sends a decision letter to the authors.
Most journals use one of three review models. In single-anonymized review, you know who the authors are but they don’t know your identity. In double-anonymized review, neither side knows the other’s identity. In open review, everyone’s identity is visible, and some journals publish the reviews alongside the final article. The model your journal uses will be specified in the invitation.
Before You Accept: Check for Conflicts
Before agreeing to review, ask yourself whether you have any conflict of interest. Financial conflicts are the most obvious, like receiving funding from a company with a stake in the paper’s findings. But non-financial conflicts matter too: reviewing a close colleague’s work, having a competing paper under review, or holding strong intellectual commitments that could bias your judgment about the topic. Career advancement, personal relationships, and institutional interests all count. If any past, current, or expected interest creates a real risk of clouding your judgment, decline the invitation and let the editor know why.
First Read: Understand the Big Picture
Read the paper once without taking detailed notes. Your goal is to absorb the research question, the approach, and the main findings. Ask yourself: What problem is this paper trying to solve? Does the research question matter? Is the overall argument coherent from introduction to conclusion?
This first pass also helps you gauge whether the paper falls within your expertise. If you realize mid-read that the methods or subject area are outside your competence, it’s better to contact the editor early than to submit a review you’re not qualified to give.
Second Read: Evaluate the Methods
The methods section is where most serious problems live. On your second, closer reading, assess whether the experimental design is robust enough to support the paper’s claims. Key areas to examine:
- Sampling and controls. Are the sampling methods appropriate? Are there sufficient control experiments? In time-dependent studies, are there enough data points to support the trends the authors describe?
- Sample size and power. Is the sample large enough to detect the effects being claimed? Did the authors address statistical power?
- Reproducibility. Could another researcher replicate this work from the description provided? Are materials, procedures, and analytical steps described in enough detail?
- Research design fit. Does the methodology actually answer the research question? For qualitative research, look for valid research questions, detailed methodology, and systematic data analysis.
Critical flaws at this level, like insufficient data or a design that can’t answer the stated question, often mean the paper needs to be rejected regardless of how well it’s written.
Checking the Statistics
You don’t need to be a statistician to spot common red flags. Start with the basics: Are the statistical methods clearly stated? Are the results interpreted correctly? Then look deeper.
Watch for over-reliance on p-values. A p-value doesn’t measure how likely a hypothesis is to be true, and it doesn’t capture the size or importance of an effect. Papers that base their entire argument on whether results cross a significance threshold without reporting effect sizes or confidence intervals are using statistics poorly. Also look for signs of selective reporting: results that seem cherry-picked, multiple comparisons without correction for false discoveries, or conclusions that don’t match the actual data. Pre-registration of the study is a good sign that the authors planned their analysis before seeing the results, which reduces the risk of data dredging.
If the statistical analysis is beyond your expertise, say so in your report. Editors appreciate honesty about the limits of your evaluation and can seek additional statistical review.
Assessing the Argument
Beyond methods and statistics, evaluate the paper’s intellectual contribution. Is the research question well motivated? A strong paper explains clearly why this specific study was needed and how it advances the field. Check whether the literature review is thorough and whether the authors acknowledge relevant prior work.
Look at whether the conclusions follow logically from the results. Authors sometimes overstate their findings, claiming causation from correlational data or generalizing beyond what their sample supports. Compare the claims in the discussion and abstract against what the data actually show. Also check for internal consistency: do the numbers in the text match the tables and figures? Are sample sizes reported consistently throughout?
Writing Your Review Report
A well-structured review report has three main parts: a summary, major comments, and minor comments.
Summary
Open with a brief summary of the paper in your own words. This signals to the editor that you understood the manuscript well enough to provide a quality review. It also helps the authors, because if your understanding doesn’t match their intentions, that’s valuable feedback on its own. This is also a good place to note the paper’s strengths, which might otherwise get buried under your criticisms.
Major Comments
Major comments address issues that could change the paper’s conclusions or that require significant additional work. These might include flaws in the study design, unaccounted confounding variables, inappropriate statistical analyses, or conclusions that aren’t supported by the data. Number each concern and explain it clearly. Where possible, suggest a path forward: what additional analysis, clarification, or restructuring would address the problem.
For example, a major comment might read: “I have concerns about unaccounted confounding. The authors adjust for age and sex but not for socioeconomic status, which is strongly associated with the exposure variable. I would recommend the authors either include this as a covariate or explain why it was excluded.”
Minor Comments
Minor comments cover smaller issues: unclear wording, missing references, labeling errors in figures, typos, or suggestions for improving readability. Separating these from major comments helps authors prioritize their revisions and keeps the review from feeling like an overwhelming undifferentiated list. If you have a long list of typos, you can group them at the end and note that the authors don’t need to respond to each one individually.
Keeping Your Review Fair and Constructive
Your comments should be objective and grounded in the paper’s scientific merits, not influenced by who the authors are, where they’re from, or whether you agree with their theoretical perspective. Bias in peer review is pervasive and often goes undetected. Reviewers tend to favor authors from the same country or gender, and intellectual biases can lead you to judge a paper more harshly simply because its conclusions challenge your own work or beliefs.
The practical antidote is awareness. Before submitting your review, reread your comments and ask whether you’d write the same thing if the paper came from a different lab or a different country. Use a respectful tone throughout. Remember that you’re evaluating the work, not the people. Even when recommending rejection, frame your feedback constructively so the authors can improve the manuscript for submission elsewhere.
Confidentiality is equally important. Don’t share the manuscript’s contents, discuss its findings with colleagues, or use ideas from the paper for your own purposes before it’s published. If you need to consult a colleague for specific expertise, disclose this to the editor first.
Making Your Recommendation
Most journals ask you to choose from four categories. Accept without changes is extremely rare for a first submission. Accept with minor revisions means the paper makes a solid contribution but needs small clarifications or corrections that won’t require another round of review. Major revisions means significant issues need to be addressed and the revised paper will go through another round of review. Reject means the paper has fundamental problems that can’t be fixed through revision, or it simply isn’t a good fit for the journal.
Your recommendation should align with your written comments. If your report lists three major methodological concerns, a recommendation of “minor revisions” will confuse the editor. Think of it this way: if the authors successfully addressed every comment you raised, would the paper be publishable? If yes, that points toward major or minor revisions depending on the scope of work involved. If you’re not confident the problems can be resolved, rejection is appropriate.
How Long a Good Review Takes
Plan for the review to take more time than you expect. The active work of reading, evaluating, and writing typically requires 3 to 6 hours, sometimes more for complex papers. Journals generally give reviewers 2 to 4 weeks to submit their report, though turnaround expectations vary. Authors perceive the typical time from submission to first decision as about 14 weeks, while most consider 6 weeks optimal. Submitting your review promptly is one of the simplest ways to support the scientific community, since delays at the review stage are the most common bottleneck in publication.
Spreading your work across at least two sessions helps. Reading the paper on one day and writing your report on the next gives your thinking time to settle, and you’ll often catch issues on the second pass that you missed the first time.