Redesigning the Blended Synchronous Peer Review Session

Decorative image

Click here for a 4 min. audio clip from a synchronous peer review session involving three students who used emailed drafts and live audio/video feed via WebEx.

Twenty-first century college classrooms increasingly occupy digital spaces in which digital citizenship skills such as using online technologies to provide and act on positive, constructive peer feedback (ISTE Student Standard 4b) are intertwined with cognitive skills such as selecting technologies to strategize and achieve learning goals (ISTE Student Standard 1) and using those technologies to interact with students from another location within a digital learning space (ISTE Student Standard 7).

Digital learning spaces may involve applications used by students who meet in a face to face (F2F) classroom. Digital learning spaces may also link students synchronously, so that students meet in real time through audio or video feeds or content creation technologies, or may link students asynchronously, such that students collaborate or engage in content creation at independently set times prior to a deadline.

This post describes the challenge of enabling first-semester college writing students to engage in effective peer review sessions in a blended synchronous environment. In the blended synchronous environment my institution has launched in its “Global Classroom” initiative, students in a class may be any or all of: a F2F group with 1 to 1 computers, individuals joining by web-conference through individual computers, and groups joining through a single web-conferencing feed from a remote site. As a technology leader at my institution, I’ve worked with other faculty to define learning challenges specific to the blended synchronous environment, along with the affordances and constraints of the institutionally prescribed learning management system and web-conferencing platform.

In my own practice as a writing instructor, I work to build student competencies in the typical learning outcomes for first-semester writing (such as employing rhetorical knowledge, applying genre and disciplinary conventions, and using sources and evidence) in part through the collaborative and social aspects of writing (WPA outcomes statement). Students use collaboration throughout the invention, planning, and drafting processes; and as they share design approaches. Students also engage in peer review sessions to revise penultimate drafts.

Peer review can enhance individuals’ ability to diagnose problems in their own work as well as provide more sustained, targeted, and interactive feedback that instructor feedback alone. Students who can work effectively together can develop student-led solutions to common writing problems and review of writing concepts from a corresponding writing unit. Peer review reinforces the mindset of working in a “community of practice,” in which genre- and discipline-specific writing goals and conventions are developed, and in which skill and knowledge construction (both of writing practices and of the problems engaged and addressed through writing) takes place collaboratively, over time and in engagement of real writing tasks, thus deepening individual skill and knowledge building and transfer.

However, first-semester students who come from diverse backgrounds may be hesitant to share their writing with others or may not possess well-developed collaborative skills. Student reticence about collaborative revision can be amplified in a blended synchronous environment, one in must also communicate through screens and microphones across distance and use technologies to engage in writing and revising. Yet employing synchronous technologies may enable students to be more active in their use of peer collaboration (Bower, 2011).

To address peer review and synchronous technologies together as a fruitful opportunity for deepening collaborative learning, I added a sixth unit to my five-unit first-semester writing unit. This unit allows for an iterative approach to peer review in which students gradually become designers of peer review strategies and employ choice in selecting associated technologies.

This unit uses the “backward design” principles of Wiggins and McTighe (2005) to structure the learning of peer review practices as a unit which takes place across the scope of the semester on non-consecutive days near the end of each of the course’s five writing units. The spread-out nature of the unit allows students time to consolidate knowledge and to apply cumulative individual and group knowledge. The “backward design” emphasis began with identifying the cognitive and metacognitive learning objectives, as well as identifying supporting technology skills, that will support effective peer review in this learning environment; from there, criteria for measuring skill and knowledge development were developed. From there, a learning environment and instructional approach was developed to support novice college writers in a blended synchronous learning format in using collaboration and technology to re-envision their writing, resulting in substantially improved drafts and a learning experience in which students gain a deeper theory and practice of what it means to revise.

Wiggins and McTighe’s (2005) three stages of “backward” instructional design

UbD Stages

Click here for a detailed description of Wiggins and McTighe’s (2005) unit design template 

 

Considerations

An important consideration for this project involved equity of outcomes. All students do not access the course in the same manner, so there is not complete technological parity in their experiences. Yet, while it may be impossible and even undesirable to require all students in such a learning space to engage in exactly the same collaborative setting, all students need equal opportunity to engage in the same learning and in the same depth of collaboration.

Technology considerations revolved around the core question of how technology can to move from being a potential or actual impediment to collaboration and improved revision skills, to supporting those outcomes.

These considerations included the broader question of which technologies can effectively promote student collaboration resulting in effective revision; the question of how to effectively use institutionally prescribed technologies; and the question of how to incorporate student input regarding technology choices and positive, respectful student engagement through technology (ISTE Student Standard 2).

Pedagogical considerations included an understanding of the research base for collaborative and constructivist learning approaches, specifically as related to writing studies. The practice of peer review, though widely used and anchored in a number of theoretical perspectives, lacks conclusive empirical proof of effectiveness. Yet, in both my practice and my study of the theoretical basis for peer review, I have found that peer review “can involve students not only in using evaluative criteria, but in processing and internalizing what makes for good writing, … sustaining deep invention and deep revision” (Farrier, 2016, p. 8). As I approached addressing the constraints and affordances of the blended synchronous learning environment in re-conceptualizing peer review as such an effective mode of learning, I wanted to be able to define and demonstrate both effective collaboration and effective revision.

 

Stage 1 of “Understanding by Design” Unit Development Process: Indentifying Learning Outcomes

Understandings

  • Feedback in the form of verbal and written response to author-, reviewer- instructor-identified writing needs (based on the demands of outcomes and of a particular writing situation) is not “criticism” but is instead a useful means of “re-visioning”: (1) a writing task, (2) a current state of a piece of written work, and (3) a plan for improvement.
  • Such improvement includes, but is not limited to, greater development and clarity of organization and expression, greater conformity to audience expectations regarding content and authorial voice, greater focus.
  • While an instructor can provide critical guidance, peers can provide guidance that is different, appropriate, and also critical for success, as in Vygotsky’s (1987) Zone of Proximal Development.
  • Providing peer feedback to someone else can help the student reviewer become better at self-assessment and self-feedback.
Essential questions

  • How does peer review fit into an individual’s writing process?
  • What is the nature and purpose of peer feedback?
  • What are effective peer feedback strategies, and How can peer feedback be provided to optimally help a peer improve a piece of writing?
  • How can a writer act on productive feedback to a work in progress?
  • How can providing peer feedback improve a students’ own writing?

 

 

 

 

Knowledge: Students will know…

  • That the purpose of a first draft is not perfection but to establish a basis for revision, which is often the stage in which the real accomplishment of a writing task occurs
  • What it feels like to make significant, effective revision, such as: greater development and clarity of organization and expression, greater conformity to audience expectations regarding content and authorial voice, greater focus
  • That the interpersonal and intellectual effort involved in supporting other writers’ work can significantly improve the writing of all members of the group; i.e., that writing is a social as well as an individual act of knowledge-construction
Skills: Students will be able to…

  • Listen to the needs of a peer writer and balance feedback such that it both addresses those needs and provides additional diagnostic cues and suggested solutions
  • Evaluate a peer’s work in progress against course outcomes, assignment criteria, and holistic rubrics
  • Provide specific suggestions for improvement through both writing and speaking
  • Incorporate peer feedback into their own revision processes
  • Flexibly select peer review strategies and purposes to best meet the writing needs of the individuals in the peer review group
  • Select and use synchronous and asynchronous technologies to support peer review

 

Stage 2 of “Understanding by Design” Unit Development Process: Determining Assessment Evidence

Summary of Assessment Evidence

Performance Tasks

  • First drafts of major papers (Writing Prompts 1-3) submitted prior to peer review sessions
  • Submitted evidence (plus instructor observation) of peer feedback
  • Second drafts of major papers submitted after peer review sessions

Criteria

  • The learning outcomes for the three major performance tasks (Writing Prompts 1-3; capstone papers for each of five units) are described in terms of outcome indicators (levels of achievement) on holistic rubrics
  • One set of outcome indicators (organization) remains fairly stable from performance task to performance task (from one unit capstone paper to another); this allows for measure of growth in addition to measure of achievement.
  • The criteria for peer review include the writing elements addressed (ranked from surface to global depth); quality of critiques and depth of feedback (ranked using the RISE model)

Other Evidence

  • Conversations with students inside and outside of class, as well as instructor observation and documentation was used to anecdotally track student self-reporting and provide formative assessment.

Self-assessment/Reflection

  • Students complete and discuss results of surveys of satisfaction and efficacy of different peer review procedures.
  • Students complete and discuss self-assessment of their peer review performance based on the peer review rubric
  • Students use and apply their evaluation of both the technologies/procedures/design of peer review session and of their learning goals to become the designers of a peer review session that they design to meet their own writing needs.
Detailed Assessment Criteria

Key Outcomes for Performance Tasks (Major papers)

Writing Prompt 1 Rubric

Writing Prompt 2 Rubric

Writing Prompt 3 Rubric

Organization

  • Employ sophistication (more than one levels of complexity or simultaneous principle of organization)
  • Use academic writing conventions to organize an introduction (provide context and/or preview the paper organization, and provide thesis)
  • Create a logical progression of ideas from paragraph to paragraph
  • Organize paragraphs with topic sentences, logical progression of connected ideas, and focus on topic
  • Use transitions between paragraphs to link paragraph ideas
  • Use academic writing conventions to organize a conclusion (extend thinking, recontextualize, and/or address unanswered questions)

Rhetorical awareness

  • Employ academic conventions (and/or conventions appropriate to audience/discourse community) of voice/stance (e.g. tone, formality)
  • Employ academic conventions (and/or conventions appropriate to audience/discourse community) of content (e.g. “What counts as evidence”)
  • Evaluate sources’ use of conventions of voice/stance and of choice of content as appropriate for their discourse communities

Key Outcomes for Quality of Peer Review Interaction

Peer Review Rubric

  • Depth of writing elements addressed (ranked from surface to global depth)
  • Depth and specificity of feedback (ranked using the RISE model)

 

Stage 3 of “Understanding by Design” Unit Development Process: Designing Instruction

Envisioning and Engaging in Peer Review: 4 Day Unit

Instructional days occur on the penultimate day of four consecutive writing units.

Summary of “Understanding by Design” WHERETO methods:

Where: Students’ prior knowledge/experience is considered in the performance task designs, which are tailored to this student population; students understand/take ownership of learning objectives though engagement with a “community of practice”-oriented learning environment in which recursive and collaborative work is normative in the classroom/digital learning environment

Hook/Hold: Student engagement for class sessions begins with either brief multimedia (such as the use of images and video in the Day 1 orientation) or with student writing or discussion. Learning of peer review practices is contextualized in terms of students’ own work.

Equip/Experience/Explore: Students are equipped to use discussion methods and any technology tools through explicit instruction. In the case of learning technology tools (such as for Peer Review Session 2), students experience a lesson involving modeling, demonstration, printed quick guides, and individual or paired experience working through the skill. Students are involved not only in conducting peer review but in evaluating the effectiveness of their individual and group work and eventually in designing a peer review session.

Rethink/Revise: The central objective of this unit is re-structuring of the revision practices that students entered the course with and transfer of acquired revision and collaboration skills.

Evaluate: By evaluating others’ writing, students develop their ability to evaluate and revise their own writing. By receiving and acting on peer feedback, students deepen their ability to evaluate and revise their own work. By evaluating the concept and practices of peer review, students engage in further metacognition and experience in tailoring their own revision practices.

Tailored: See Evaluate and Where (above). Assignment of group members takes into consideration individual students’ learning needs.

Organized for engagement: By structuring learning of peer review practices as its own unit  which takes place across the scope of the semester on three non-consecutive days near the end of each course unit, students have time to consolidate knowledge and to apply cumulative individual and group knowledge.

Day 1 (Peer Review Session 1 Using Reading, Discussion, and Handwritten Feedback)

Materials: Scaffolding for Peer Review Session 1, Writing Prompt 1 Rubric, Peer Review Rubric

Orientation/Equipping

Prior to Peer Review Day:

  • Contextualize instruction in specific objectives for the writing assignment (a literacy narrative) in the prior instruction during the corresponding writing unit. For example, students read literacy narratives by student and professional writers and examine the narrative structures chosen by writers as they develop their own narrative structure (organizational plan), which will be evaluated during peer review.)
  • Contextualize orientation to the nature of revision (as opposed to editing) in the earlier parts of the writing unit as students work in an environment that emphasizes a recursive process, for example, by writing paragraph-length exercises and revising them.
  • Present a mini-lecture on peer review practice as a professional practice, including a visual of the peer review process for publication, a checklist of the professional benefits of such peer review, and a possible video. Referring back to professional peer review practice as well as frequent, focused (brief) use of peer work/collaboration during subsequent instruction equips students to expect and practice collaboration.
  • Have students visit and discuss the “history” and “edit” pages of a Wikipedia article to see the collaborative editing process at work and conceptualize online collaboration.

On Peer Review Day:

    • Review with students how to use a rubric as a revision guide.
    • Review and discuss the Peer Review 1 instructions. Suggest a time frame that allows 20 minutes per person and that the groups establish additional norms and expectations beyond the peer review guidelines.
    • Implement Scaffolding for Peer Review Session 1

Experience/Exploration

  • Student groups engage in peer review. I allow for 1.5 hours of a 3 hour class. This includes transition time to get oriented. I also allow student groups who are meeting in person (though this is a blended synchronous class, there may be more students who meet in person than who join remotely) to leave the classroom and find a comfortable space (such as a hall nook with armchairs or a table in the student center) to work. I visit groups to monitor progress.

Evaluation

  • Following the peer review session, the whole group discusses their experience. I guide the discussion to include feedback from each individual. This allows for individual self-assessment, which will be extended after the next peer review session. Students also receive feedback from me verbally (such as affirmation of a chosen approach or suggestions for how to better accomplish a specific peer review task next time), through written, criteria based feedback via rubric, and through the modelling I provide in my own written and verbal feedback on each student’s work.
Day 2 (Peer Review Session 2 Using Google Docs)

Materials: Scaffolding for Peer Review Session 2 (including Getting set up to share and edit files on Google Docs), Writing Prompt 2 Rubric, Peer Review Rubric, and ENG 121 Reflection 2

Orientation/Equipping

  • Contextualize instruction in specific objectives for the writing assignment (a close analysis) in the prior instruction during the corresponding writing unit. For example, students grapple during the writing unit with the problem of constructing a thesis which is a statement about another author’s work and with organizing a close analysis according to topics rather than by re-telling a work of literature; thesis statements and paragraphs organized around topic sentences will be evaluated during peer review.
  • Teach the technology: Prior to the peer review session, students watch the instructor model use of Google Docs, then work individually or in a pair through the process of uploading and sharing their work on Google Docs.
  • Review and discuss the Peer Review 2 instructions. Suggest a time frame that allows 20 minutes per person and that the groups establish additional norms and expectations beyond the peer review guidelines.

Experience/Exploration

  • Student groups engage in peer review. I allow for 1.5 hours of a 3 hour class. This includes transition time to get oriented. Students remain in the computer lab (or at their remote sites) to work. I monitor progress.

Evaluation

  • Following the peer review session, the whole group discusses their experience. I guide the discussion to include feedback from each individual.
  • Students also complete a survey in which they evaluate their experience with and the effectiveness of each of the two peer review sessions/formats they have engaged in. I then lead a brief discussion in which students articulate what they value about peer review and make suggestions for Peer Review 3. I then construct Peer Review 3 to reflect students’ preferences and choices (design).
    • The survey results indicated that:
      • While most students liked feeling that that they could edit “at their own pace” when using Google Docs (as opposed to having to write suggestions on a draft while listening to an author read as in the first session), most found the verbal interaction involved in the live reading more conducive to greater depth of revision
      • Students felt that there was a temptation to give less effective feedback after the first page of a Google Doc draft, and this was in part because it was difficult for a whole group to stay oriented on one part of the draft, especially given difficulties with staying connected via audio given the technology options for this class
      • Students unanimously wanted more critical constructive feedback from their peers.
  • Students also receive feedback from me verbally (such as affirmation of a chosen approach or suggestions for how to better accomplish a specific peer review task next time), through written, criteria based feedback via rubric, and through the modelling I provide in my own written and verbal feedback on each student’s work.
Day 3 (Peer Review Session 3 Using Student Choice)

Materials: Scaffolding for Peer Review Session 3, Writing Prompt 3 Rubric, Peer Review Rubric

Orientation/Equipping

  • Contextualize instruction in specific objectives for the writing assignment (a rhetorical analysis) in the prior instruction during the corresponding writing unit. For example, students grapple with the concepts (such as audience, context, exigence, constraints) involved in learning to view writing rhetorically. Students also grapple with how to effectively integrate quotations and paraphrases.
  • Because the many writing concepts students encountered during the unit are still being worked out in their writing, I choose 5-8 key concepts and writing “problems” that are typical of students’ drafts and present 5-8 “Tips and Tricks” via PowerPoint or multimedia before the Peer Review session. For example, one tip involves rewriting a sentence structure to eliminate a mistake most students made when introducing their sources. Another tip reviews three academic strategies for concluding a rhetorical analysis.
  • Student groups are requested to come up with at least one “Tip or Trick” to share with the whole group following the peer review session.
  • Review the student choices and goals that informed Peer Review 3 and discuss the Peer Review 3 instructions. In this version of the course, students preferred live interaction.

Experience/Exploration

  • Student groups engage in peer review. I allow for 1.5 hours of a 3 hour class. This includes transition time to get oriented. Students chose to have groups read all papers in the group and note suggestions for revision, then use face to face or synchronous discussion with video to have each writer read his/her work aloud, stopping after each paragraph for group discussion.
    • As indicated in the audio recording of one group’s interaction, the group was organized and engaged; each paper was thoroughly reviewed; students had specific goals for improving their work that were addressed by the group. I noted more attention to sentence- and paragraph-level issues than to global issues, such as overall depth and organization. However, I address those issues in my feedback; whereas student feedback allowed for much more thorough revision (not mere editing) at the paragraph level and addressed questions of voice and sentence structure more thoroughly than my feedback alone can.

Evaluation

  • In a post peer review discussion, students use the Peer Review Rubric to evaluate their own feedback, then discuss the effectiveness of the peer review session. I share my observations as formative assessment toward students’ development of effective peer review practices (See Experience/Exploration, above).
  • Students groups share their “Tips and Tricks.” This allows for sharing of common writing problems, review of writing concepts from the corresponding writing unit, and collaborative, student-led generation of solutions to writing problems. It reinforces the mindset of working in a “community of practice,” in which genre- and discipline-specific writing goals and conventions are developed, and in which skill and knowledge construction (both of writing practices and of the problems engaged and addressed through writing) takes place collaboratively, over time and in engagement of real writing tasks, thus deepening individual skill and knowledge building and transfer.

 

Day 4 (Peer Review Session 4 Using Student Choice)

Day 4, a repeat of Day 3, allows for a final student-designed and self-evaluated peer review  session and demonstration of mastery of peer review, before students embark on the fifth and final writing unit. For Day 4, students and instructor may opt to stay with the third peer review plan as designed by students, or may engage in yet another iteration.

 

Reflection

One of the most valuable aspects of the “Understanding by Design” model, and one which directly addresses the key issue, for writing studies, of transferability (Yancey, Robertson, & Taczak, 2014) is the authors’ discussion of the multi-facted nature of “understanding” (Wiggins & McTighe, 2005). Wiggins and McTighe explicitly reference writing instruction as instruction that encompasses a range of six types of understanding (p. 84):

  • Explain: via generalizations or principles, providing justified and systematic accounts of phenomena, facts and data; make insightful connections and provide illuminating examples or illustrations
  • Interpret: tell meaningful stories; offer apt translations; provide a revealing historical or personal dimension to ideas and events; make the object of understanding persona or accessible through images, anecdotes, analogies and models
  • Apply: effectively use and adapt wha we know in diverse and real contexts
  • Have perspective: see and hear points of view through critical eyes and ears; see the big picture.
  • Empathize: find value in what others might find odd, alien, or implausible; perceive sensitively on the basis of prior direct experience
  • Have self-knowledge: show metacognitive awareness; perceive the personal style, prejudices, projections, and habits of mind that both shape and impede our own understanding; are aware of what we do not understand; reflect on the meaning of learning and experience

While I enjoy teaching writing precisely because it involves multi-faceted types of understanding, it’s easy to fall back on teaching the types of understanding that I am most proficient with or find easiest to teach.

The opportunity in this unit to focus specifically on developing students’ peer review skills has given me new opportunities to think about how to help students develop more difficult-to-teach abilities such as empathy and self-knowledge. Three key insights for me have been the attention that I must give to defining and teaching specific revision and collaboration skills; the way a spread-out unit taught throughout a course can give students time to develop, conceptualize, and apply complex writing understandings, such as those dealing with rhetorical awareness and global organizational patterns; and the way that a blended synchronous learning environment may actually have affordances that can enhance student learning.

A breakthrough for me was the use of Emily Wray’s RISE Model to clarify my criteria for teaching and assessing peer review collaboration, and to help me develop a rubric that could be used for formative assessment and student reflection as well as for summative assessment. 

Although I have used the concept of a spread-out additional unit before, I was more satisfied with the success of this unit, doubtless because of the extent to which the unit’s instruction was contextualized in the writing units that preceded peer review days, but also because of the heightened focus provided by the “Understanding by Design” attention to learning outcomes in the form of thoroughly considered “understandings.”

Finally, I was able to tackle the constraints of the blended synchronous learning environment, an environment for which research on design and implementation factors is just beginning (Bower, Dalgarno, Iennedy, Lee, & Kenney, 2015), and to discover some of the affordances of the environment (Bower, 2011). I believe that one of these is the opportunity to focus on and experiment with student interaction and another is the opportunity to solicit student input in peer review design and implementation. Blended synchronous meetings take more time than classroom peer review activities may. A peer review session in which several students from two or more locations interact through video conferencing and share and revise manuscripts through email or collaborative composing technologies necessitates the time and patience required for students to become comfortable with interacting and to become familiar with the skills and performance criteria involved in peer review. At the same time, students develop group expectations and take the time to provide extended feedback on writing problems that matter to the individuals in the group. Students in my blended synchronous peer review sessions received far more feedback and took more time to act on feedback than if they had simply received instructor feedback.

Synchronous technologies provide opportunity for students to develop not only respectful and responsible communication skills (ISTE Student Standard 2b) for peer review but to move into the realm of learning as a co-emergent process in which students and teachers collaborate together, a subject I’ll address in my next post.

In the first implementation of this unit, I found that because students brought different experiences and skill sets with technology to the table, that knowledge or even lack of knowledge could be added productively to my own in designing peer review practices. Because of the resulting greater ownership by students of the instructional process, higher achievement in writing also resulted.

 

References:

Bower, M. (2011). Synchronous collaboration competencies in web-conferencing environments — their impact on the learning process. Distance education, 32(1), 63-83. https://www.tandfonline.com/doi/abs/10.1080/01587919.2011.565502

Bower, M., Dalgarno, B., Kennedy, G.E., Lee, M.J.W., & Kenney, J. (2015, March). Design and implementation factors in blended synchronous learning environments: Outcomes from a cross-case analysis. Computers & Education, 86, 1-17. Retrieved from https://www.sciencedirect.com/science/article/pii/S0360131515000755

Colorado Community College System. (2018). Common Course Numbering System ENG 121 Required Syllabus Information. [PDF document.] Retrieved from: https://www.cccs.edu/wp-content/uploads/documents/ENG-121-Required-Syllabi-Info-CO1.pdf

Farrier, S. (2016). Principles for the Practice of Peer Review. (Unpublished paper). University of Colorado, Denver.

Vygotsky, L.S. (1978). Mind in society. Cambridge, Massachusetts: Harvard University Press.

Wiggins, G. & McTighe, J. (2008). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.

WPA outcomes statement for first-year composition (3.0). (2017, July 14). Retrieved from http://wpacouncil.org/positions/outcomes.html/

Yancey, K. B., Robertson, L., & Taczak, Kara. (2014). Writing across contexts: Transfer, composition, and sites of writing. Logan: Utah State University Press.

 

Leave a Reply

Your email address will not be published. Required fields are marked *