A process for developing and validating performance assessment rubrics for laboratory techniques

Abstract

PROBLEM

Achieving competence in technical skills is a key learning objective in the laboratory programs of our institution’s Pharmaceutical Science course. However, their assessment is often achieved through indirect outcome indicators since performance assessment through direct observation is time-consuming and there is a lack of validated, efficient and scalable methods.

PLAN

We report on our approach towards the development, validation and optimisation of a set of rubrics for the formative and summative assessment of common chemistry laboratory techniques. Efficiency and user-friendliness are key criteria in the rubric development. 

Drawing on recent work by Seery et al. (2017) and Chen et al. (2013), we propose a four-stage process:

1. Rubric design based on practitioner input, literature sources and lab observations
2. Testing involving student self-assessment, peer evaluation and instructor assessment
3. Optimisation and validation based on assessment data, student and instructor feedback
4. Developing supporting procedures (e.g., instructor training).

ACTION AND EVALUATION

We have started implementation in two synthetic chemistry subjects, initially targeting rubrics for four techniques (reflux setup, thin-layer chromatography, vacuum filtration and recrystallisation).

Rubric evaluation will involve both quantitative (assessment data) and qualitative methods (surveys/focus groups with instructors and students, human ethics application in progress).

REFERENCES

Chen, H. J., She, J. L., Chou, C. C., Tsai, Y. M., & Chiu, M. H. (2013). Development and application of a scoring rubric for evaluating students’ experimental skills in organic chemistry: An instructional guide for teaching assistants. Journal of Chemical Education, 90(10), 1296–1302. https://doi.org/10.1021/ed101111g

Seery, M. K., Agustian, H. Y., Doidge, E. D., Kucharski, M. M., O’Connor, H. M., & Price, A. (2017). Developing laboratory skills by incorporating peer-review and digital badges. Chemistry Education Research and Practice, 18(3), 403–419. https://doi.org/10.1039/C7RP00003K