Assessing critical evaluation skills of undergraduate science students: developing expertise in science communication

Abstract

Background The ability to critically evaluate and use evidence from one’s own work or from primary research literature is invaluable to any researcher. But the benefits of these skills go beyond the needs of scientific research, as the ability to create coherent scientific arguments is based on one’s ability to support claims with evidence (Bugarcic, Colthorpe, Zimbardi, Su, & Jackson, 2014; Toulmin, 2003). The skills required to critically evaluate and use evidence include the ability to identify strengths and weakness of primary literature; to gauge the impact of research findings on a field; to identify gaps in a field that require more research; and to contextualise findings within a field (Blommel & Abate, 2007). Methods This study examined the development of undergraduate science students’ abilities to critically evaluate and use evidence, through an analysis of the discussion sections of their written laboratory reports from an inquiry-based laboratory class. In order to gauge these skills, a model of epistemic levels was developed to analyse the way in which students use evidence to support their claims. In addition, the discussion sections of published scientific research articles were subjected to the same analysis, and comparisons made between student and expert authors. Results The student reports analysed (n=42) used evidence in a variety of ways, using 6.6+/-0.2 of the nine categories identified in the model. Most often referring to literature indirectly (29%), and least commonly highlighting limitations of literature (2%). The reports averaged 25.5+/-1.9 instances of evidence use across 762+/-35 words. The number of references cited varied from 5 to 23, averaging 10.1+/-0.5 per report. There were significant positive correlations between grade and number of references, length, and number of instances of evidence use, with these two latter variables also being strongly correlated. When compared to students, instances of use of evidence in scientific research articles (n=7) were very similar, averaging 7.3+/-0.3 of the nine categories of evidence use. However, expert authors were significantly more likely to refer to their own results than students, with this being the commonest (37%) type of evidence use, and cited significantly more references. Conclusions The inquiry-based curricula provided opportunities for students to development their skills in critical evaluation of evidence use, and the construction of arguments based on their own findings. Analysis of their laboratory reports showed that students, by the completion of the second year of their undergraduate degree program, already have expertise approaching that of published authors. These findings demonstrate that it is possible to provide valuable broad-scale undergraduate experiences to all students in a cohort, giving them exposure to the methods and communication processes of research, and an opportunity to hone their critical evaluation skills. References Blommel, M. L., & Abate, M. A. (2007). A rubric to assess critical literature evaluation skills. American Journal of Pharmaceutical Education, 71(4), 63. Bugarcic, A., Colthorpe, K., Zimbardi, K., Su, H. W., & Jackson, K. (2014). The Development of Undergraduate Science Students’ Scientific Argument Skills in Oral Presentations. International Journal of Innovation in Science and Mathematics Education, 22(5), 43-60. Toulmin, S. E. (2003). The uses of argument: Cambridge University Press.