Development and validation of a concept inventory for introductory-level climate change science

Abstract

This paper follows on from Jarrett, Takacs and Ferry (2011) which reported the first stage in development of a concept inventory (CI) for the science of climate change. This paper details the process of CI item development, reports statistical results of initial field trials and outlines how these will be used to further refine the CI. Item difficulty, discrimination, and point biserial coefficient were calculated for each item. Cronbach’s alpha and test-retest data were used to assess reliability. Results suggest that about half of the items were too difficult for high school students; although item discrimination and test reliability were close to acceptable values, which suggests that most students were not simply guessing answers. Although it was initially designed for use in high schools, a group of undergraduates trialled the CI. Statistical analysis of scores suggest that for this group, the items perform better, and well within acceptable values. Given these favourable results and the fact that introductory-level climate change is an increasingly taught at universities, further trials with undergraduates are taking place. It is intended that the final CI will be made available as a formative assessment instrument. The draft version is available from the authors on request.