Standards in assessment

The attempts to define standards for computer based assessments have so far been largely unsuccessful. I think that one of the problems is the lack of clarity in the functional domain. Do we really understand the ontology of an assessment, and a question? I don't really think we do, and perhaps we never will. It is easy enough to find a way to define a multiple choice question in XML, but to do the same for 'any' question... I think it's a bit much to ask. You inevitable end up constraining what you can do.

This was one of the main problems with IMS QTI 1.2. The specification was incredibly limited, and thus any systems supporting the standard by definition were as limited. Worse, most systems did not even implement the standard fully, or correctly, and so QTI 1.2 never really got anywhere.

Version 2 was supposed to solve this. The specification (currently still a draft, version 2.1) is indeed a lot better, and allows for much more questions types, feedback, and scoring strategies. The problem is that to make all this possible in a standard XML definition, the specification has gotten rather complicated. I'm not sure it is a viable proposition to expect any vendor to support the standard in full. To make matters worse, all the big vendors, but also the Open University's OpenLearn, seem to be pushing the Common Cartridge, which includes an amended version of IMS QTI. 1.2. While it would be nice to be able to exchange and run questions that are embedded in learning materials from Blackboard or Moodle, it it does strike me as very unlikely that any vendor will now have a serious incentive to support anything beyond the Common Cartridge.

And so we might have to live with the fact that we are not going to have any standard for the exchange of question and/or assessment information. I'm not sure that s a bad thing though. We would probably be better of designing a decent system first, in stead of trying to standardise functionality that hasn't even been implemented anywhere yet. What use is interoperability, if there isn't anything to exchange?