Comparing comments from novice peer reviewers against comments from subject-matter experts on the same papers, this study asked what types of comments both groups actually found helpful.
Cho, K., Schunn, C. D., Charney, D. (2006). "Commenting on writing: Typology and perceived helpfulness of comments from novice peer reviewers and subject matter experts." Written Communication, 23(3), 260–294.
Anecdotally, students often assume peer comments can't match an expert's. This study tested that assumption directly by comparing the types of comments student peer reviewers produced against comments from an independent subject-matter expert on the same papers, then asking which types both groups actually found helpful.
The researchers collected comments on classmates' papers from two undergraduate and one graduate-level psychology course, with an independent, experienced psychology instructor also commenting on a subset of the same papers. Comments were classified by type and rated for perceived helpfulness.
Undergraduate peers consistently found directive comments (ones that suggested a specific fix) and praise comments helpful. Critically, the writing expert also endorsed the helpfulness of the same directive comments — peer and expert judgment converged on what actually counts as a useful comment.
This is direct evidence that peers and experts aren't applying different, incompatible standards for what "good feedback" looks like — they agree on the comment types that work. That convergence is part of why aggregated peer feedback can be trusted at all: it isn't a different, lesser kind of evaluation, it's the same evaluative judgment an expert would apply, just distributed across more people.
Chris Schunn co-authored this study on where peer and expert judgment about feedback quality converge. He is a Professor of Psychology, Learning Sciences and Policy, and Intelligent Systems at the University of Pittsburgh and a Senior Scientist at the University's Learning Research and Development Center (LRDC), where he has directed research projects backed by more than $80M in federal grants. As Peerceptiv's Chief Learning Scientist, his research directly shapes how the platform structures reviewer prompts, rubrics, and feedback workflows.
