Experimenting with open peer review in class

https://thelukewarmersway.wordpress.com/2016/02/07/
climate-scientists-in-like-flint/

The previous posts were all written as part of a series of course assignments with the aim to experiment with open education and open peer review / open evaluation. Students were given 2 guidelines for writing a blog post and reviewing / elaborating their feedback to others' blog posts.


Here are the guidelines for the blog structure:

1. Very brief abstract, 1 paragraph, containing the main question and an outlook to its answers.
2. Description of motivation and context of question, including respective seminar literature or anecdotes.
3. Answer/s to the question/s
4. Implications/discussion of answers
5. Brief description of methods to get answers (including challenges and failures)
6. Outlook what else one would like to know / do
7. Further materials (pics, vids, interviews, …) and links (since this is a course on open science, think about making your materials as openly accessible as possible!)
8. In between: pics or quotes (from your text, materials, or famous people…)

Audience

You are targeting students, like yourself, but the blog should also be understandable to people without any prior knowledge of Open Science and STS!

OPEN PEER REVIEW
And here are the required points for the review exercise (besides detailing decisions in a written feedback):

General Factors Ratings

Please provide a rating from one star (poor) to five stars (excellent)

• Level of importance: Is the publication of relevance for the seminar and the target community of the blog, and does it provide important insights? Does the work represent a novel perspective or even new findings?
• Level of validity: Is the core question/concern clearly formulated? Is the argumentation stringent? Is the interpretation balanced and supported by sources?
• Level of completeness: Do the authors reference the appropriate scholarly context (e.g. refer to seminar literature)? Do the authors provide or cite all necessary information to follow their findings or argumentation? Do they miss relevant publications in the field?
• Level of comprehensibility: Is the language correct and easy to understand for the broad audience of the blog? Are images or figures well displayed and captions properly described? Is the article systematically and logically organised?

The experiment also led to an interesting discussion of the limits of openness in peer review. Which level of openness do we feel comfortable with in relation with the topics of our posts and our roles as students in higher education? What are our benefits, what are the drawbacks?