Building the capacity of academics to assess higher order skills through improved assessment design

Abstract

Across the sciences, high-stakes assessment (such as end-of-semester exams) often consist largely, if not solely, of multiple choice questions (MCQs). MCQs have advantages in large-scale testing, including automated marking and high reliability (Haladyna 2004). But in biomedical education, MCQs usually test knowledge recall (the lowest “Remembering” category in the cognitive domain of Blooms taxonomy). Psychometric analysis of MCQs exams in biochemistry and physiology from four universities has shown that they contained a substantial number of questions that were too easy for the cohort and lacked questions testing higher-order skills such as problem-solving, and knowledge analysis and application . It is possible to design MCQs that test higher-order cognitive processes—the UMAT and GAMSAT medical entrance exams routinely include MCQs which test higher-order cognitive skills. But to write questions of this standard requires skill and effort. The objectives of this workshop are: To build academic capability to design and write MCQ exams that test students’ capacity to solve problems and apply their knowledge/competencies in new contexts; and to improve the validity of assessment by adopting a rational assessment design approach, including mapping of questions against subject learning objectives.