How Students Use the Scientific Writing Assistant Chatbot to Support Scientific Writing

Osu Lilje; Tsz Wai Rosita Pang; Claudia Keitel; Januar  Harianto; Monica Basuki; Matt Pye; Christopher Hammang; Michael Widjaja; Fran van den Berg; Ryan Jones; Doris Lin

Authors

Osu Lilje University of Sydney https://orcid.org/0000-0003-4770-0103
Rosita Pang University of Sydney https://orcid.org/0000-0002-9900-572X
Claudia Keitel University of Sydney https://orcid.org/0000-0003-3101-7529
Januar Harianto University of Sydney https://orcid.org/0000-0002-4803-108X
Monica Basuki University of Sydney https://orcid.org/0000-0002-6682-3021
Matt Pye University of Sydney https://orcid.org/0000-0002-0470-9848
Christopher Hammang University of Sydney
Michael Widjaja University of Sydney https://orcid.org/0000-0003-1509-0443
Fran van den Berg University of Sydney https://orcid.org/0000-0003-1666-9422
Ryan Jones University of Sydney
Doris Lin University of Sydney

Keywords:

scientific writing, Chatbot feedback, student engagement

Abstract

SUB-THEME: Technology Enhanced Learning

BACKGROUND

This study explores how students engage with the Scientific Writing Assessment (SWA) Assistant—an AI-guided feedback tool designed to support scientific writing—in the context of BIOL1007, a large first-year biology unit with over 1500 students. The SWA Assistant was developed as part of a shift from take-home to in-class scientific report assessments, aiming to help students refine core skills such as hypothesis testing, experimental design, and data analysis. This development is grounded in a conceptual framework that emphasises ongoing engagement with the scientific process (Krest & Carle, 1999; Lilje et al., 2008a,b).

AIMS

The main research aims addressed in this article are to investigate how the SWA Assistant supports student learning outcomes in scientific writing and to explore its potential in enhancing scientific literacy across diverse student cohorts.

DESCRIPTION OF INTERVENTION

The SWA Assistant is an AI-guided feedback tool that enables low-stakes, iterative feedback, allowing students to identify whether their writing meets assessment criteria and where improvements are needed (Jerde & Taper, 2004).

DESIGN AND METHODS

Between 2023 and 2025, we analysed over 4500 student records and 1543 SWA Assistant conversations involving 721 unique users. Student queries were categorised into themes such as feedback interpretation, writing clarity, concept application, and integration of evidence.

RESULTS

Results indicate that the assistant encouraged autonomy, fostered deeper engagement with scientific writing, and may have contributed to academic integrity by reducing reliance on external content generators.

CONCLUSIONS

These findings offer insights into how AI can enhance scientific literacy equitably and effectively across diverse student cohorts.

Author Biographies

Osu Lilje, University of Sydney

Dr. Osu Lilje (Senior Lecturer; School of Life and Environmental Sciences, Faculty of Science) - Responsible for overseeing the overall direction and strategy of the research project, providing guidance and expertise to ensure its successful completion. (EF Academic)
Rosita Pang, University of Sydney

Dr. Tsz Wai Rosita Pang (Sessional Academic, Teaching Fellow in Biology and Chemistry, School of Life and Environmental Sciences, Faculty of Science) - Primary contributor to research project, ethics application, grant application, data collection, data analysis, and manuscript preparation. (Casual EF Academic)
Claudia Keitel, University of Sydney

Dr. Claudia Keitel (Senior Lecturer in Biology, School of Life and Environmental Sciences, Faculty of Science) - Assists with data collection, data analysis, and manuscript preparation. (T&R Academic)
Januar Harianto, University of Sydney

Dr. Januar Harianto (Lecturer in Agricultural Data Science & Biostatistics, School of Life and Environmental Sciences, Faculty of Science) - Assists with data collection, data analysis, and manuscript preparation. (EF Academic)
Monica Basuki, University of Sydney

Monica Basuki (Sessional Academic, School of Life and Environmental Sciences, Faculty of Science) - Assists with data collection, data analysis, and manuscript preparation. (Casual EF Academic)
Matt Pye, University of Sydney

Dr. Matt Pye (Senior Lecturer in Biology, School of Life and Environmental Sciences, Faculty of Science) - Assists with data collection, data analysis, and manuscript preparation. (EF Academic)
Christopher Hammang, University of Sydney

Christopher Hammang (Educational Designer, Faculty of Science) - Assists with data collection, data analysis, and manuscript preparation.
Michael Widjaja, University of Sydney

Dr. Michael Widjaja (Lecturer in Biology, School of Life and Environmental Sciences, Faculty of Science) - Assists with data collection, data analysis, and manuscript preparation. (EF Academic)
Fran van den Berg, University of Sydney

Dr. Fan van den Berg (Lecturer in Biology, School of Life and Environmental Sciences, Faculty of Science) - Assists with data collection, data analysis, and manuscript preparation. (EF Academic)