In high school AP English classes, peer review reached reliability and validity comparable to expert grading — when the rubric was written in language students could actually use.
Schunn, C. D., Godley, A. J., DiMartino, S. (2016). "The reliability and validity of peer review of writing in high school AP English classes." Journal of Adolescent & Adult Literacy.
Reliability findings from college-level peer review don't automatically generalize to high school students or to the specific demands of AP-level writing instruction. This study asked whether structured peer review could hold up to the same reliability and validity standard in an AP English classroom, and what rubric design choices made the difference.
The researchers examined peer review of student writing in high school AP English classes, comparing the reliability and validity of the resulting peer ratings against the standard set by expert grading.
Peer review of writing in AP English classes reached reliability and validity comparable to expert assessment. The rubrics that made this work were student-friendly and focused directly on the specific skills the course was designed to teach, rather than generic writing-quality criteria.
A rubric borrowed generically from another team or another company rarely works as well as one written in plain language around the exact skills your own program is trying to build. If peer review isn't producing reliable results, the rubric's specificity and clarity are the first place to look before concluding peer review itself doesn't work for your context.
Chris Schunn co-authored this study on what makes a peer-review rubric actually work with real students. He is a Professor of Psychology, Learning Sciences and Policy, and Intelligent Systems at the University of Pittsburgh and a Senior Scientist at the University's Learning Research and Development Center (LRDC), where he has directed research projects backed by more than $80M in federal grants. As Peerceptiv's Chief Learning Scientist, his research directly shapes how the platform structures reviewer prompts, rubrics, and feedback workflows.
