Assessment of learning outcomes workshop: How do you know that your assessment tasks are effective?

Abstract

Background Higher Education in Australia is in a phase of rapid change due to regulatory changes (TEQSA) and a shift towards a standards based framework. Over the past five years, the Chemistry community in Australia has developed the Chemistry Threshold Learning Outcomes (CTLOs) which articulate the outcomes that every student graduating from an Australian university with a major in Chemistry will have attained. In keeping with this development, the Royal Australian Chemical Institute (RACI) now bases its accreditation for Chemistry degrees on the CTLOs. Therefore, it is now vital to the Chemistry community to ensure that the assessment items we use allow students to demonstrate attainment of all CTLOs during their chemistry degree. Our OLT funded project (Assessing the assessments: Evidencing and benchmarking student learning outcomes in Chemistry (OLT ID14-3562)) has developed a diagnostic framework that will help you to determine whether your assessment items actually deliver reliable measures of student performance and provide evidence of achievement of the CTLOs. An additional outcome of the project will be a database of standards-based assessment items to be shared with the Chemistry community. Workshop Format We invite you to attend this half-day workshop where project team members will guide you through evaluating your assessment items for their ‘fitness for purpose’ in providing evidence of achievement of the CTLOs. Ideally you will bring along one of your 2nd or 3rd year Chemistry assessment items (in electronic format) so that the workshop team can guide you through an online submission and evaluation to determine: 1. Which CTLOs are explicitly demonstrated by students through successful completion of the item? 2. Is your task suited to a developing or graduate level understanding? 3. To what extent can your task be said to help confirm student attainment of the CTLOs? A critical aspect of this process is consideration of marking schemes and student work for the assessment items. We strongly encourage you to bring one or two pieces of marked pass level student work (de-identified) that can be used to evidence successful attainment of a particular CTLO. A central part of the workshop will be a ‘calibration’ exercise, which allows developing a mutual understanding of what constitutes demonstrating attainment of a particular CTLO. While the content of this workshop is Chemistry specific, the process is not. Therefore we are confident that all ACSME attendees will find this to be a valuable experience. All are invited. Attendance is free, however, registration is required for catering purposes. Proceedings of the Australian Conference on Science and Mathematics Education, The University of Queensland, Sept 28th to 30th, 2016, page X, ISBN Number 978-0-9871834-4-6.