Peer Reviews

Before your submission is graded, you'll need to complete two peer reviews of others' submissions. You'll also have the opportunity to do informal peer reviews during lectures and labs, on your peers drafts.

Giving and receiving useful peer review is hard. Here's some guidance on each.

Giving

When you give peer review, you should first be clear about what the author(s) want from you. Are they asking you to evaluate their submission against the grading rubric? Or are they seeking some more specific guidance about a particular aspect of their work? If they haven't specified what they want, you should ask them, so you can focus your feedback. (If you're conducting a formal peer review of a submitted work, then your job is already specified: grade against the rubric).

In addition to what to peer review, there are also many ideas for how. Your feedback should be: 

• Constructive. It shouldn't just criticise, but also talk about what is good about the submission and how it might be improved. Provide concrete suggestions about what could change.
• Detailed. It should be specific about what is good and bad, not generic. Give examples of what made something work or not work, and in some cases multiple examples. Giving vague platitudes or critiques does not help the author(s).
• Justified. It should detail why things are good and bad, not just that they are. It can be hard to articulate this sometimes, but this internal rationales are something authors deserve to know about your thought process. You may also find that when you interrogate your thinking, you reconsider your feedback.

Remember that all of your peers are growing their confidence in writing and argumentation, and your role in this process is to help them grow it. Be kind, loving, and supportive, but also firm about your critiques.

If you're doing formal peer review (as part of a submission), we'd like you to format your feedback as follows:

Analysis

Coherence (score/4.0)

Provide a justification for the score, aligned with the rubric. Consider aspects such as:

• Is the argument logical, or are their inconsistencies, logical leaps without clear connections, and disconnected ideas?
• Are the claims in the argument clearly represented by topic sentences, or they hard or impossible to find?
• Do the claims the argument demonstrate insight about the information system, or are they superficial?

Use of Evidence (score/4.0)

Provide a justification for the score, aligned with the rubric. Consider aspects such as:

• Are the claims well substantiated by some kind of evidence, such as lived experiences or  credible primary sources, or are there unsubstantiated claims?
• Are the sources used credible, or are they from suspect sources with motives that call into question whether the claim is true?
• Does the evidence invoked enrich the argument by strengthening its authority, or does it weaken the argument because their lack of credibility or relevance?

Lived Experience (score/4.0)

Provide a justification for the score, aligned with the rubric. Consider aspects such as:

• Is the author's use of their own lived experience used as a source of authority or insight, or is it superficially connected to the topic?
• Does the lived experienced mentioned help deepen the analysis of the system, or does it merely serve as context?

Final Score (score/4.0)

Explain how you combined the three scores above to arrive at the final score, noting which issues had the greatest weight in your judgement. This is not necessarily an average of the scores. For example, a 4.0 should generally be strong in all areas, but doesn't necessarily need to be a 4.0 in every area; perhaps the area it was weakest in was a minor issue that didn't detract from the overall strength. And vice versa: a submission may be strong in two areas, but exceptionally weak in another, justifying a lower score.

Design

Problem (score/4.0)

Provide a justification for the score, aligned with the rubric. Consider aspects such as:

• Do you understand the problem and its causes?
• Are you persuaded by the evidence presented that the problem is actually a problem, or is the evidence presented superficial and/or lacking credibility?
• Is it clear who experiences the problem, and what other stakeholders are affected by it?

Solution (score/4.0)

Provide a justification for the score, aligned with the rubric. Consider aspects such as:

• Are the details of the proposed solution sufficiently clear that you could imagine how it could work, or is it so vague, you can't imagine it?
• Was the description sufficiently complete that you felt all significant aspects of the system's ability to address the problem were discussed, or were there elements not discussed that make it unclear how it would address the problem?
• Were the details clear, or where there ambiguities that made it unclear which of many possibilities the designers(s) intended?

Tradeoffs (score/4.0)

• Did the submission detail who would benefit and who would be harmed, or was this omitted or vague?
• Were the harms it discusses plausible and specific, or just vague general categories of harm without any particular connection to the solution proposed?

Final Score (score/4.0)

Explain how you combined the three scores above to arrive at the final score, noting which issues had the greatest weight in your judgement. This is not necessarily an average of the scores. For example, a 4.0 should generally be strong in all areas, but doesn't necessarily need to be a 4.0 in every area; perhaps the area it was weakest in was a minor issue that didn't detract from the overall strength. And vice versa: a submission may be strong in two areas, but exceptionally weak in another, justifying a lower score.

Receiving

When you're seeking feedback, the first and most important thing is to clearly tell your peer what you want feedback about. Don't just say that you want "feedback". Do you want them to grade your submission against the grading rubric? Do you want feedback on a particular section of your writing? Do you want feedback on some particular aspect of your submission, such as its tone, or credibility of its claims? If you don't provide a concrete request, the person giving you feedback can't give you concrete feedback.

Once you receive feedback, remember a few things about it:

• It won't always be clear. Sometimes you'll need to ask for clarification, or if you can't, you'll have to interpret what someone might have meant. That can be frustrating, but communication is hard.
• It may be hard to hear. Sometimes we can see our writing and ideas as extensions of ourselves. It's important to remember that they aren't. We may have created them, but in communicating them, they become a public phenomena, something that is for others to react to. Listen to how they reacted to see how others are seeing and interpreting what you said. You may have to say it differently to get your point across more clearly.