Paper Review · Psychology and social science
A pre-submission review for psychology papers.
Choose the psychology and social-science profile at upload and the four reviewers check the things replication-minded referees now ask about first: pre-registration, forking paths, samples and effect sizes.
The psychology and social-science profile
What the reviewers look for
These are the instructions the profile adds to the review, in plain words. Each finding in the report quotes the passage it is about and says what would fix it.
- Pre-registration. Whether the study was pre-registered, and whether the analysis reported matches the plan; predictions that read as hypothesized after the results were known.
- Forking paths. Many outcomes and moderators with only the significant ones reported, and whether robustness or multiverse checks cover the researcher's choices.
- Samples. WEIRD samples generalized to everyone, and MTurk or Prolific samples treated as representative.
- Effect sizes. Small effects presented as practically meaningful.
- Measures. New scales used without reported reliability; alphas under 0.7 are flagged.
- Design checks. Manipulation checks present and working, order effects and counterbalancing disclosed.
- Confounds. Demographic variables left uncontrolled.
- Replication. Whether there is independent replication, or this study is the only evidence.
What a finding looks like
An example from the report
Illustrative, not from a real manuscript: this is the shape each finding takes. The full sample report shows every section.
The reported analysis departs from the pre-registration.
Quoted: "As predicted, the effect was moderated by trait anxiety."
Surfaced by: Methodology Critic, Editor-in-Chief (consensus).
Fix: The pre-registered plan lists anxiety as an exploratory covariate, not a moderator. Label this analysis exploratory, or report the pre-registered model first.
In every review
The rest of the report
- Four reviewers: a Methodology Critic, a Statistical Skeptic, a Data Integrity Officer and an Editor-in-Chief, with findings more than one of them raise listed first.
- Every reference checked against CrossRef, and for the most important claims, the cited paper's abstract read to see whether it supports the sentence citing it.
- A figure and table scan for integrity and presentation problems.
- An anonymity scan for double-blind submission.
- A prioritized checklist, as a Markdown report and an annotated PDF with comments on the pages they refer to.
- The Journal Pack ($11) adds a check against one venue's submission rules. In psychology and social science it covers Psychological Science, Journal of Personality and Social Psychology, Cognition and Academy of Management Journal. All 25 venues.
Questions
Does it work for qualitative or mixed-methods papers?
How do I choose this profile?
What happens to the manuscript?
How long does it take and what does it cost?
Read it before Reviewer 2 does.
From $9, usually back in 4 to 8 minutes. No account, no subscription.
Other profiles: Machine learningBiomedicineChemistry and materials