INTERACTIONS WITH AI: CAN STUDENTS CRITIQUE AND MODIFY AI-GENERATED OUTPUT?

Abstract

Generative artificial intelligence (AI) is now a part of the technological toolkit with which students need to be proficient for their future employability. Our students, however, have varying experience with generative AI tools. Part of our role as educators is to create educational opportunities where students are taught how to critique and adapt output from generative AI programs like ChatGPT. As part of a larger project to rethink assessment across all first-year biology units of study at the University of Sydney, we created an assessment where students needed to critique and modify an introduction for a scientific paper, written by Chat GPT. Based on evaluations from a similar assessment last year, we added in scaffolding exercises that allowed them to practice and receive feedback on the critique process. Overall, the scaffolding exercise improved students’ ability to critique generative AI output, however, many students did not fully translate their critique into modifications of the ChatGPT generated introduction text for their own assessment. This intervention demonstrates that despite directives from many universities to allow students to use generative AI as a co-pilot for their assessments, first year science students need to be explicitly taught how to critique and modify generative AI output as part of our transition pedagogy.