Evaluating the New Technologies

Abstract

There is an accelerating trend towards the use of new technologies in teaching; the challenge is to demonstrate its effectiveness. In approaching the evaluation of new methods, what were the aims of the initiators? While their over-riding expectation or hope is usually to enhance learning in a specific area, other possible outcomes may include reaching more students, providing experiences otherwise impossible, offering flexible access to a wider range of information, encouraging rehearsal and practice in virtual environments, the development of more generic skills - including the use of computers themselves. Unless these expectations are made explicit, the impact or effectiveness of the technological solution cannot be measured against its own goals. Ongoing evaluation in use can subsequently feed into quality improvement cycles. In other situations, comparative judgements are sought but in many ways this approach is difficult. There is by no means agreement on the best methods of evaluation, even for the most basic of questions: Is the technology more effective in enhancing students' learning than are the alternatives it replaces? Is it cost-effective? Is it received better by the students? Conventional teaching methods have by no means always been evaluated rigorously, so the baselines for comparison are lacking or flawed. Complex variables (characteristics of the program in which the technology is embedded, students, teachers) inevitably confound any differences found, so absolute judgements are rarely possible. Nevertheless, some strategies for evaluation have evolved and are often very effective within a local context. Examples from fixed media and web-based technology will be discussed.