How generative AI can enhance personalised feedback at scale

Abstract

Educators play a crucial role in a student’s learning journey by providing assessment feedback that is personalised, encouraging, constructive and timely (Hattie et al., 1996; Black & Wiliam, 1998 and Hattie & Jaeger, 1998 as cited in Carless, 2006). However, they often struggle with heavy workloads and the skill to provide consistent, quality feedback that is meaningful to students. On the other hand, students, especially those new to written assessments, can find feedback to be unclear or overly critical, which can be discouraging and confusing, hindering their learning process. These challenges are magnified in large-cohort units of study where the marking load is done by large numbers of casual staff members leading to issues around standardisation and budgetary constraints.

To address this challenge, we built an AI assistant using Cogniti, a generative AI tool created by educators at the University of Sydney. Cogniti allows educators to steer large language models (LLMs) like GPT-4 with specific instructions and resources, producing tailored outputs that support student learning. The Cogniti agents, designed as either a biology expert or data scientist, aid in delivering constructive feedback on written assessments. These agents helped markers to improve and expand their written feedback on scientific reports for two first year units of study: “Introduction to Statistical Methods” and “From Molecules to Ecosystems” in the School of Life and Environmental Sciences, with large enrolments of up to1400 students. Importantly, markers-maintained control over the entire marking process, drawing upon their disciplinary knowledge and experience to ensure precise grading.

We describe the impact for markers and unit coordinators in using Cogniti as a feedback expander and demonstrate the types of feedback it can provide. In addition, we outline how easily Cogniti can be cloned and modified so that it can be implemented in new units of study. The dual time saving approach in being able to improve the quality of feedback in a limited amount of time and be easily built and adapted for new units of study, makes Cogniti an ideal tool for improving feedback in large-cohort units of study.