About CATs

Douglas and Hegelheimer (2008) gave a nice review of the developments in the use of computer technology in language assessment, and there were two particularly interesting points in their discussion I would like to reflect on here. First, as what the authors have concluded, it seems that most argument and debate in this area could be boiled down to "appropriate and inappropriate uses of CATs", which mirrors the result of most of our classroom discussion; and "what was once thought to be the wave of the future in language testing, that is, computer-adaptive tests, has been tempered by the realization of the limits of this particular technology", which reminds us the importance of not getting overwhelmed by the ever-developing technology and throwing away comparatively traditional but effective methods and practice.
 

Second, the article mentions a research conducted by Cohen and Upton (2006)( as cited in Douglas & Hegelheimer, 2008), which has found that "in general, the test takers tended to approach the test tasks as test tasks rather than as genuine reading tasks. In other words, they were mainly interested in getting the correct answer rather than learning anything from the reading". Thinking back, I have had the same problem when I was doing the internet-based test practice to prepare for TOEFL when I constantly needed to resist the temptation to click on the "checking results" button to get instant grading for my answer without really caring about what I could learn from the reading itself. However, on second thought, I would have done the same thing if I could flip the paper and check my answer in pen-and-pencil language tests. Therefore, is this a limitation of CATs only or a limitation for all forms of tests in general. Because, in my opinion, once something is turned into a test, it is human nature for people to focus on finishing the task correctly and pass the test instead of the so-called deep thinking, regardless of whether it is computer-aided or not.

Douglas, D., & Hegelheimer, V. (2007). Chapter 6: Assessing language using computer technology. Annual review of applied linguistics, 27, 115-132.