Case studies

Subject areas:

Sue Taylor – Accounting

Sue uses a student peer review-based assessment task in her second year combined Accounting and International Business class, International Accounting. The original peer review task was conducted in-class and was staff-led and administered but has now been moved to the PRAZE system which provides a very friendly, e-learning inter-face for staff and students.

Why did you decide to use student peer review?
This subject has large classes – up to 400 students per year. These students are both culturally diverse and are drawn from a range of disciplines – for example, accounting, international business, finance and management. Many students also have English as a second language.

While this diversity presents challenges in terms of the development of the curriculum, it also means that the students themselves are one of the best learning resources available to the class. Our responsibility then is to develop assessment tasks that allow the students to draw on the different experiences and international heritage of their peers in order to better inform their understanding of the subject content.

The adoption of a student-based, peer review task was introduced to facilitate this knowledge transfer process and to provide additional feedback and support for time-poor students in terms of their understanding of the key issues. It was also hoped that by developing collaborative, effective feedback practices within a peer review format, the quality of student participation and engagement would improve. It was also hoped that students would be better able to monitor and evaluate the quality and impact of their own work.

Could you describe the assignment for which peer review was used?
Originally, the in-class, peer review task (10% of the assessment) was designed around the completion and marking of weekly, mathematically-based, tutorial tasks and was totally staff-led and administered. The students completed the work at home, bought their answers to the tutorial in the week following the lecture and then these answers were randomly allocated by the tutor to other students in the class to mark – using a staff-prepared marking scheme.

An e-technology alternative to this original, in-class, staff administered, peer review process was sought and PRAZE was selected for its ease-of–use and excellent reporting systems back to the unit co-ordinator as well as its ability to provide a collaborative learning environment for students.

Given the advantages of the PRAZE system, the peer-review assessment focus was increased in significance in terms of a new focus on the major (30% of the assessment), individual, written assignment. This assessment task required the students to complete a comparative analysis of international, corporate, annual reports, focusing on the cultural impacts on business and accounting systems and practices across Anglo Saxon, Germanic/Latin, Asian and Middle Eastern countries.

The PRAZE system’s flexibility and ability to deal with complex documents and to relieve staff of almost all administrative responsibilities, while providing a very friendly inter-face for students, were key reasons for this increased peer-review focus.

What were the benefits to you as an administrator and to the students?
When I originally introduced the in-class, staff-administered peer review assessment task, the failure rate dropped from 27% to 6%. There was 98% attendance – tutes were packed. The students were so engaged some staff members complained they were asking so many questions it was difficult to get any other work done! The administrative process for staff was also very time-consuming taking staff away from their other academic commitments – such as research projects for example.

With the PRAZE system handling almost all of the administration process, the staff involved in the subject had no administrative load at all. As the Unit Co-ordinator, other than loading up the student files and handling minor student queries, I also was relieved of significant levels of administrative responsibilities with the PRAZE system handling the setting up of assignments, the specification of classes and groups and the establishment of distribution rules.

The average mark in the cross-cultural project went from a high pass to a distinction. Students completed their work early as they were awarded marks for the loading up into PRAZE of their assignment draft. They could work together on-line via the PRAZE system, share ideas via their peer reviews – drawing on the strength of the cohort, and “creating a village atmosphere” (although I did other activities as well to encourage this). It helped develop the lifelong learning and team skills of the students, all the generic skills we try to develop.

These anonymous, peer-review communications between the students were also found to be far less threatening than those coming from staff (often seen as supervisors or authorities). This was a particularly important advantage for students with English as second language along with the ability to actually see examples from peers of how to structure work and demonstrate critical thinking. This latter skill can be a difficult thing to achieve for some students given their cultural background.

How did students find participating in the peer review?
As reflected by the significant increase in the average mark, the students were very engaged in the assessment task and voted in multiple, anonymous and voluntary surveys to retain the PRAZE system of peer review with a 100% approval rating. Because of the way the project was structured, students could work collaboratively to master the basic frameworks at the draft stage obtaining valuable feedback and support from each other via the PRAZE process prior to submitting the final version of the project for formal grading.

Examples of student comments:

“Seeing another high quality draft helped steer me in the right directions because it made me think critically about my own concepts and ideas”

“Peer review made me think of the task questions and how appropriately I had actually answered my questions”

“The reviewer’s comments were very useful. Any uncertainties I had were clarified by a fellow student doing the same task”

What advice would you give to someone trying out peer review?

• Be conscious of language barriers – some domestic students devalued feedback from ESL students because of a lack in English proficiency.
• Set clear, specific questions in the beginning to guide the peer review.
• Be aware that students are time poor – I set no more than 3 pages to be reviewed, and students only did 2 reviews each.
• Students need scaffolding/support in terms of how to both actually write a peer review, and how to effectively process (reflect on) the feedback received, which may be seen as confronting/critical.

Gavin Buskes – Engineering

Gavin has used PRAZE in his first year Engineering subject, Engineering Systems Design 1 for approximately 3 years.

Why did you decide to use student peer review?
Students originally had to do a weekly reflection piece. With 700 students in the course, this was very cumbersome to mark. So we began using student peer review to assess the journals instead.

I originally started using PRAZE as a tool to facilitate the management of the journal marking. Using the Learning Management System (LMS) was very intensive, especially for late submissions; it couldn’t do what I needed it to do. The main advantage of PRAZE was that it took all the management and automated it, and could also handle proxy submission straight to the system (which helped a lot with late submissions).

Could you describe the assignment for which peer review was used?
Students complete a reflective journal. Over the course of the semester, they submit this for feedback, and perform 3 reviews of other people’s work. I provide a review form rubric, so students rate the piece according to these categories.

Originally, students gave marks for the reviews, but they complained about the potential corruptibility. Now, they just provide reviews, and staff members do the actual marking at the end of semester. Students get marks for participating and performing reviews, so if they don’t want to and are happy losing the marks associated with it, they don’t have to participate.

Students also use the group self assessment to rate contributions to group work from themselves and the others in their group 3 times per semester. They have to justify why they give their ratings. Project marks are then scaled according to participation, although this doesn’t actually modify the marks too much – it just lets students feel there’s a mechanism for review without creating a big skew. If there’s a big difference between self and peer assessment, I call the group in to discuss the ratings. People are fairly honest though, so this is rare.

View the ‘Engineering systems design’ review form (PDF 186 kB).

What were the benefits to you as an administrator and to the students?

• Collection and distribution of work from all the students is hassle free, so the administration is much easier.
• There’s been a big reduction in the number of complaints about group work and people not pulling their weight.
• It teaches generic skills that they don’t otherwise get in Engineering, like self-reflection, reviewing work, teamwork and so on.

How did students find participating in the peer review?
It’s popular with students – it provides extra feedback for their work, and they feel there are mechanisms for making sure everyone contributes. If they don’t want to participate in the peer review, they don’t have to.

What advice would you give to someone trying out peer review?

• If I were trying to sell it to other staff members, I’d sell it as a tool for admin, because it handles that really well and that’s an easy sell to people who aren’t convinced about generic skills or peer review. It also makes the students happier and raises the subject’s SES scores.
• A lot of subjects though should take this on for the sorts of skills (teamwork, reflection) students can learn through it.
• I’ve made changes to the system over the years to try to improve things – bringing in the group self assessment, changing the way students are marked for feedback.
• My main gripe is that the peer review isn’t automatic. I can set it up to autorelease, but generating distributions must be done manually, so I have to leave an hour between closing submission and opening the reviews to remember to do it.

Helena Bender – Environments

Helena uses PRAZE’s group work evaluation functionality in Reshaping Environments, a first year subject in the Bachelor of Environments degree. This subject attracts a very broad student cohort from 11 different majors. In previous years, she used the full PRAZE system, but dropped back to just the group work evaluation recently.

Why did you decide to use student peer review?

• Builds the students’ skills in critiquing, self-reflection and self-improvement; gets them away from doing their final assessment at the last minute and the “first draft is the best draft” mentality.
• Reduces the expectation of feedback from the tutor.
• I made the change to just the group work evaluation because I didn’t feel the feedback provided through peer review was adequate. It had very high time demands for both students and tutors, and was hard to assess the peer reviews.

Could you describe the assignment for which peer review was used?
Originally, students conducted an audit (in terms of water, energy, biodiversity and waste) of a building of their choice from on campus as a group project. This audit then underwent peer review from other groups, who were primarily critiquing the approach the audit took, rather than their findings.

In the current assignment, students all evaluate the one site, thinking about 2 proposals for that site, and evaluate which would be the most sustainable option as a group project. They begin this in Week 5, then discuss group progress in tutes in Week 7 (as a dry run for the final evaluations they give in Week 10, to provide those who might not be contributing fully time to catch up).

What were the benefits to you as an administrator and to the students?
For the students, it raises their confidence. It also increases competence – skill building. It provides them with a framework for providing feedback, to help them structure critiques of their own work (self-reflection). It also gets them to start early and keeps them on track.

I may be going back to using the full peer review system, with a smaller project that will go through a full peer review. It will still get them to start early, but they will be providing feedback rather than tutors. I will provide models for them to see what constitutes a Pass and what constitutes an High Distinction. This may be more cost effective than the current project as well.

What advice would you give to someone trying out peer review?

• Students must submit on time – the peer review can’t function otherwise. Make sure you plan for what to do if they don’t submit their projects or their reviews on time, so you know what the fallback/consequences are.
• You need to make sure the review questions provide clear guidance so the students don’t misdirect each other in feedback and ensure there aren’t too many questions.
• Have a discussion prior to the review – discuss “what is a critique?”, not just “This is good” or “This is poor”. Get the students to think about the sort and level of feedback they would want on their work.
• Make sure they provide positive feedback.

Melanie Plesch – Music

Melanie has used PRAZE’s Group Peer Assessment tool in a first-year core subject, Medieval and Early Modern Music, for three years.

Why did you decide to use student peer review?
A key assessment component in this subject was a research- and performance-based group project. I used peer assessment to factor a combined mark for the different group assignments. The combination mark also provided a fair reflection of individual effort. There is often an intrinsic unfairness in group work in that some people can ride on the work of others, and peer review helped prevent that.

Could you describe the assignment for which peer review was used?
Working in groups, students explored a concept or an idea through the four periods of music history covered in the subject (Medieval, Renaissance, Baroque and Classical), and created a five-minute collage composition based on a suite of relevant works, which they performed at a concert in Melba Hall at the end of semester. The project consisted of two written assignments (a topic proposal and a full director’s score) and the final performance. Peer review was performed after each assignment. Individual grades were arrived at by combining the mark received by the group (60%) with the mark given to each student by his/her peers (40%).

What were the benefits to you as an administrator and to the students?
This subject had large classes (150 +). Using PRAZE made the administration of the peer assessment component much easier. With the group project students were much more engaged than in a traditional essay/exam assessment mode; attendance at lectures and tutorials improved and they were constantly talking about their project. Peer assessment provided students with a sense of ownership of the assessment process and encouraged them to be responsible for their own learning.

How did students find participating in the peer review?
Students invariably comment favourably on the peer review exercise: they felt that it prevented other students from taking a “free ride”.

“I was grateful for the peer-review function, because I worked very hard and it was assuring to know I would be fairly assessed compared to others who didn’t put any effort in.”

I found students to be very fair and compassionate markers and also self-critical and honest in their self-assessment.

What advice would you give to someone trying out peer review?

• Start early. Setting up an assignment takes time and things can go wrong so it is best to have plenty of time.
• Provide a framework with clear guidelines on giving and receiving feedback. Make your expectations very clear. I often found that for the first assignment students were inclined to rate everyone in their group as “excellent”, so we had to discuss in class the likelihood of all members contributing with the same level of excellence. One of the positive aspects of the exercise was that students learnt that being critical and rigorous does not mean being unkind.
• Do not despair. Learn from mistakes and use students’ criticisms and comments to improve the assignment for the next time.

Michelle Livett – Physics

Michelle uses student peer review in the first year subject Physics 2:Life Sciences and Environment.

Why did you decide to use student peer review?
In planning to use peer review I anticipated that reading and reviewing other students’ writing would improve their own as they saw how different students tackled the task. The assignment enabled students to complete a writing task that developed their capacity to construct a carefully sequenced response to a question. The peer review component was also intended to trigger students’ thinking about what is required for a short piece of writing to be effective in explaining a physics idea. I hoped that by being part of a whole class engaged in this task students would be encouraged to feel part of a group of students learning with each other.

Could you describe the assignment for which student peer review was used?
Students wrote a 200-250 word response to a question based on one of the areas of physics in the early part of the subject. They needed to explain a physical phenomenon described in one of three questions – one required a comparison of the heating of an imaginary planet and Earth by the sun; the second compared the forces required to hold a balloon and fitness ball underwater; the third question explored the area of fluid flow in the body. The students completed two peer reviews, using specified criteria. They reviewed students’ responses to the two questions they did not answer themselves, and received two student reviews as well as a review from a teaching staff member before editing and re-submitting their final answer for assessment. Their reviews and final answer were assessed.

What do you believe were the benefits to you as an administrator and to the students?
There was no doubt that students valued feedback on their own work from other students and teaching staff. Many reflected on how they learnt from the comments from others on their own work, but also on the value of seeing quite different ways that other students went about constructing their own writing. Reading other students’ writing did prompt the examination of the characteristics of quality writing as I had hoped. However the students cited that it was their understanding of physics that benefited more than their writing skills. The assignment required them to really understand an area of physics before they could write a response, and it was this deep thought and review of their own understanding that students valued most highly. This depth of learning also developed through their review of other students’ work. A level of understanding was required to critique the writing and they experienced this in all three areas of physics that were the subject of the assignment.

How did students find participating in student peer review?
Examples of student comments:

“I think it was the ability to see how others addressed different questions. It gave us another viewpoint and another way of approaching similar questions in the future.”

“The ability to not only choose the question that was to be answered, but also providing and receiving feedback to other students’ answers was an interesting and helpful experience. This is because it promoted deeper thinking and gaining insights into particular concepts from different points of view to construct further understanding.”

“More interesting than just doing an assignment and handing it in. It was more interactive and allowed us to get a perspective on other students’ work and aided in learning the ideas of other topic indirectly.”

What advice would you have for someone trying out peer review?
I recommend strong scaffolding for the task. While the details provided to students were comprehensive, the improvements I will be putting in place for next year will be to be even clearer with students about my expectations of them, in particular how strictly the word limits would be enforced.

Jenny Martin – Science Communication

Jenny used student peer review in her 2^nd year undergraduate subject Communicating Science and Technology and her Masters subject Science Communication.

Why did you decide to use student peer review?
I wanted to use student peer review for several reasons. Firstly because in the case of writing about science for lay audiences, the students themselves are just as equipped as I am to give feedback as to whether a piece of writing is engaging, easy to understand and informative. I also really wanted to give the students the opportunity to redraft their work on the basis of feedback before final submission and based on feedback from someone other than me. Finally, I wanted the students to all gain experience in giving constructive feedback since peer review is a very important part of the research process.

Could you describe the assignment for which student peer review was used?
Students are assigned the task of writing the same scientific or technological story for three different audiences (3 x 500 words; 1500 words total). The three different audiences are:

1. Scientific journal readers
2. Local newspaper readers
3. Primary school teachers

Students are required to review two other students’ work.

View the ‘Science Communication’ review form (PDF 195 kB).

What do you believe were the benefits to you as an administrator and to the students?
The students enjoyed having the opportunity to read each others’ work and became much better at reading their own work critically having assessed other students’ work. Explaining to someone else why, for example, a particular piece of writing wasn’t well targeted at a particular audience helped them to make their own writing more suitable for that audience. As an administrator, the peer review process gave the students actual experience of giving feedback (much more useful than me just teaching them about the process) and ensured that all the students got more feedback on their writing than they would have otherwise.

How did students find participating in student peer review? Any example comments or feedback would be great.
Almost all of them found it useful to some degree and were happy to have the opportunity to improve their writing before final submission. Some were frustrated that the comments from their peers were different to the feedback I gave them on their final version.

What advice would you have for someone trying out peer review?
Go for it – I think peer review should be much more widely used in education – there are many benefits for both students and staff. My only advice would be to think carefully about whether you need to have any say in which students review each others’ work. In my case, where the nuances of language were of the utmost importance for the assignment, it was very beneficial not to have a student for whom English is not their first language reviewing the writing of another student in that category.

Angus Campbell – Veterinary Science

Angus uses student peer review in the 3^rd year Bachelor of Veterinary Science subject Small Ruminants.

Why did you decide to use student peer review?

• To engage students in processes of critical thinking about a topic from ‘both ends’: as creators of material but also as reviewers of someone else’s material.
• To use a method of assignment-based assessment that helped make the marking process more efficient.
• To try something a bit different!

Could you describe the assignment for which student peer review was used?
Each subject in the 3rd year of the University of Melbourne Bachelor of Veterinary Science covers medicine and surgery of a different species. In the subject ‘Small Ruminants’ (medicine and surgery of sheep, goats, alpaca and deer), students were required to develop an imaginary scenario presenting a disease problem in one of these species. The peer review involved two things. Firstly, a student colleague reviewed the scenario based on criteria such as its realism and clarity. The same colleague conducted a ‘virtual disease investigation’, answering a set series of questions about their diagnosis of the cause of the disease in the scenario and its underlying risk factors. The answers to the investigation were returned to the student who wrote the scenario, for them to provide feedback to the investigator about the diagnostic process and whether or not the investigator’s diagnosis was correct.

The peer review assignment was worth 15% of a year-long subject, with the marks for the assignment totally determined by the peers, with moderation from the lecturer.

View the ‘Small ruminants’ review form (PDF 213 kB).

What do you believe were the benefits to you as an administrator and to the students?
There was a (slightly) reduced marking workload, although this was offset by having to check and moderate a large portion of the marks (mainly the outliers).

Students benefited by having to understand a disease enough to imagine how it would present in real life and create a realistic scenario. They also had to consider other diseases that could present with similar signs. In other words, the peer review assignment helped them to start thinking like clinicians, both from the perspective of a disease and its differentials, as well as from the perspective of performing a disease investigation themselves and assessing another student’s investigation. Developing a scenario also required students to understand farming systems well enough that they could set the disease in a realistic scenario.

Peer review provided a wide range of scenarios for students to investigate (more than if I had to think them up myself), and allowed students to practice written communication skills. I was very impressed with how encouraging most students were to others when they gave their feedback—student reviewers were often very supportive and very interested in enhancing their peers’ learning, going out of their way to provide extra information and explanation. That was probably the most gratifying part of the whole experience for me. Peer review was also creative—they weren’t writing scenarios for other students as well as the lecturer—and a bit of fun.

How did students find participating in student peer review?
There was generally very good feedback. A few students in 2011 specifically stated they didn’t like peer review, although some went on to acknowledge that was just a personal preference. Underlying reasons for dissatisfaction seemed to be associated with feeling like there was a lack of standardised marking or excessively harsh marking from the peers, even though I tried to address this by reviewing and moderating marks, where appropriate.

What advice would you have for someone trying out peer review?

• Students seem to enjoy and benefit from peer review, and peer review can potentially allow students to examine a topic from a different perspective.
• Administering peer review across a large class is a lot of work. It isn’t necessarily a time-saver compared to manually marking assignments.
• The devil is in the detail of the system that distributes and collects the assignment across the student cohort! I found keeping track of who said what to whom was confusing, probably mainly because there were two rounds of exchanging material and reviews between pairs of students. Extracting the data from the system to review and collate was a very large task that never went completely smoothly. Even once the data were extracted, I had to write quite a complex spreadsheet to compile and facilitate moderation of marks this, which was a bit of a job in itself.

Raoul Mulder – Zoology

Raoul has used student peer review in the 3rd year Zoology subject Animal Behaviour and the companion subject Experimental Animal Behaviour for the past five years.

Why did you decide to use student peer review?
My interest in peer review grew out of a sense of frustration with the way we tend to provide feedback to students in the university context. Usually that involves some comments and a mark on the final version of an assignment which means that the student doesn’t really have any opportunity to further improve upon that work. I wanted to try and five students a lot more feedback and to provide that feedback earlier. So, involving fellow students as reviewers seemed like a good idea.

Could you describe the assignment for which student peer review was used?
Animal Behaviour: Students are assigned the task of producing a succinct account of a recent publication in the style of a ‘News & Views’ piece for the journal Nature. The piece must describe the research and explain its significance to an audience of non-specialists. Each student reviews three student assignments, and receives three reviews. Each student also has the opportunity to give feedback on the quality of the review through a ‘feedback form’.

View the ‘Animal Behaviour’ review form (PDF 198 kB).

Experimental Animal Behaviour: Students work in groups of four to carry out a research project and write a report. Students choose a topic from a list of fifteen broadly pre-defined areas and then carry out their research under the supervision of a tutor. Although the research is carried out in groups, students prepare and submit individual reports. Students upload their individual draft report for distribution to three reviewers: the group’s tutors plus two student reviewers from different groups. Report authors use the reviews to imporve their final submission, and write a ‘Letter to the Editor’ in which they detail how they dealt with reviewers’ comments.

‘Experimental Animal Behaviour’ review form (PDF 203 kB)

What do you believe were the benefits to you as an administrator and to the students?
Administrator: Using peer review means that students get comprehensive feedback without increasing my marking load. Plus, using the online peer review tool PRAZE makes the distribution of assignments and reviews simple. Although I had to spend a bit of time setting up the assignment and review forms when I first began using peer review, now it’s really quick and easy.

Students: I believe that peer review is extremely beneficial to students including developing a better understanding of subject material and improving their critical thinking skills. I have found it brings some students out of their shells and involves some students that might otherwise not be very engaged. Students also realise their view point is as valid as anyone else in the class including the teacher which is empowering, and they learn to differentiate between helpful and unhelpful feedback. I have found that students are often surprised by how much they can learn from reviewing the work of others – it triggers a thought process about their own assignment which helps them to go back and improve their work. Because peer review is anonymous, it creates an open dialogue and allows students to express honest and often valid opinions.

How did students find participating in student peer review?
Students responded very positively to the peer review process. I conducted a survey in the subject Experimental Animal Behaviour which showed that the majority of students (about 80%) believed the review process had helped them to improve the quality of their final submission. This is supported by written feedback comments such as: “Reviews were a great help!”, “Reviews were a very good aspect of this subject”, “The peer review process was very helpful” and “I think the peer review exercise should be introduced into every Zoology subject”. The survey and comments indicated that most students had an extremely positive experience of peer review.

Some students expressed concern about the perceived variability in quality of reviews and indicated they would have preferred to have feedback from tutors/lecturers. Some students also commented that writing the reviews was a lot of extra work.

What advice would you have for someone trying out peer review?

• Provide detailed training to the students on how to write a good review
• Ensure that the review forms are structured around the assessment criteria. I found this is a useful way of reinforcing and reminding students of what the assessment criteria are.
• Consider including a tutor as one of the anonymous reviewers, so each student receives at least one ‘expert’ review.
• Include some sort of feedback form to allow students to give feedback to the reviewers on the quality of their reviews.
• Try to include assignment topics, and ensure students do not review their own topic. This will help prevent plagiarism.