Establishing online examination guidelines: Preliminary results from an ACDS Teaching and Learning Project 2022

Sara H. Kyne; Reyne Pullen; Siobhán S. Wills

Authors

Sara H. Kyne School of Chemistry, Faculty of Science, University of New South Wales, Sydney NSW 2052, Australia
Reyne Pullen School of Chemistry, The University of Sydney, Sydney NSW 2006, Australia
Siobhán S. Wills School of Chemistry, Faculty of Science, University of New South Wales, Sydney NSW 2052, Australia

Keywords:

assessment design, assessment for learning, Blooms Taxonomy, examinations, STEM education

Abstract

In recent years, many universities in Australia and worldwide have seen major changes to their teaching and learning delivery. This includes assessment strategies, an example of which are summative final examinations. Historically, closed-book, in-person, paper-based final examinations were commonly used across the sector (Williams & Wong, 2009). However, during the COVID-19 pandemic many universities moved from traditional paper-based examinations to online delivery (Dicks et al., 2020). Online examinations have been delivered in a variety of formats, and with different implementations. Thus, we are at an opportune time to re-evaluate assessment for and of learning to ensure that we make pedagogically informed changes and establish robust procedures moving forward.

In this research, funded by an Australian Council of Deans of Science (ACDS) Teaching and Learning Project grant 2022, we present the preliminary results of a multi-institution exploration of first-year undergraduate examinations in STEM subjects comparing end-of-semester examinations from 2019–2021. To determine the pedagogical changes that occurred, we undertook a multi-step analysis of:

i) Question type and format;

ii) Order of thinking pattern required to respond to questions (Agarwal, 2019);

iii) Classification of question according to Bloom’s Taxonomy (Bloom et al., 2001);

iv) Level of abstraction.

Outcomes from our data analysis will inform practitioners and decision-makers on best practices whilst balancing university and student expectations, with delivering authentic assessment experiences. Our research is enabling us to make meaningful recommendations for best practice in Australian STEM subjects for summative examinations, including design that considers both technological as well as pedagogical aspects required to deliver effective assessments.

REFERENCES

Agarwal, P. (2019). Retrieval Practice & Bloom’s Taxonomy: Do Students Need Fact Knowledge Before Higher Order Learning? Journal of Educational Psychology, 111(2), 189–209. https://doi.org/10.1037/edu0000282.

Bloom, B. S., Airasian, P., Krathwohl, D. R., Cruikshank, K., Mayer, R., Pintrich, P., Raths, J., & Wittrock, M. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Anderson, L. W., Bloom, B. S., Krathwohl, D. R., (Eds.), Longman: New York.

Dicks, A. P., Morra, B., & Quinlan, K. B. (2020). Lessons learned from the COVID-19 crisis: Adjusting assessment approaches within introductory organic courses. Journal of Chemical Education, 97(9) 3406–3412. https://doi.org/10.1021/acs.jchemed.0c00529.

Williams, J. B., & Wong, A. (2009). The efficacy of final examinations: A comparative study of closed-book, invigilated exams and open-book, open-web exams. British Journal of Educational Technology, 40(2) 227–236. https://doi.org/10.1111/j.1467-8535.2008.00929.x.