12. QUALITY OF USE OF MULTIMEDIA LEARNING SYSTEMS
12.3 ENGINEERING QUALITY OF USE FOLLOWING ISO 13 407
In the past, statements about the adoption of user centered design tended to be strong on general philosophical orientations, but weak on guidance as to how these orientations should be implemented in the design process. This section outlines the model proposed by the draft standard in overview, adding comments and glosses in order to bring out the relevance of this model to the usability engineering of multimedia learning systems.
1. plan the human centered process
complete ~
2. specifY the
~ context of use
5. evaluate ~ 3. specifY user
designs against and organisational
user requirements requirements
~ 4. produce design V
solutions
Figure 12.1. ISO 13 407 model.
The model comprises of five stages, four of which are implicitly joined in a loop.
This section will examine each stage in tum. Although the process outlined above looks iterative, it need certainly not be so: it may be converted to a waterfall life cycle model if required by simply going through once only (in this case, there is simply more focus on user needs and user evaluation than one would normally expect to find in a conventional system development) or a V-type life cycle development (in which the evaluation phase is seen as signing off the specification phase). However, the true benefit of this model emerges when it is used to guide an iterative development process.
184 The Digital University - Building a Learning Community 12.3.1 Plan the Human Centered Process
This first stage requires the gathering of the commitment of all concerned in the development process to the user-centered design philosophy, and to create a plan whereby there is ample time and opportunity for engaging in user requirements elicitation and testing as well as the more technical aspects of development.
The necessary side-effect of this first step should be to gain consensus among the design team that user involvement in the project is not simply at the end, that is, to
"baptize" the result with "usability evaluation". A Validation Plan is the outcome of this first stage. Such a plan specifies how many iterations will be carried out and time-lines for each. However, this plan should also list the success criteria to be reached at each stage and the methods to be adopted to attain these criteria and to check that the criteria have been reached. The BASELINE project (see Section 12.4) proposed and tested a User-Based Validation Assistant which is a large pro- forma that enables an organization to manage these concerns. Although the BASELINE User-Based Validation Assistant was designed explicitly for use in projects involved in the Information Engineering domain of the EC's Telematics Application Programme it is orientated towards industrial usage outside of this programme, in line with the general objectives of the EC's Telematics Applications Programme as a whole.
Although the Validation Plan may appear to be outside the loop, in practice, the first draft will never be the last: it is more often a working document which is first produced in outline terms and then reviewed, maintained, extended and updated during the design and development process.
12.3.2 Specify the Context of Use
The quality of use of a system depends on understanding and planning for the characteristics of the users, tasks and the organizational and physical environment in which the system will be used. It is important to understand and identify the details of this context in order to guide early design decisions, and to provide a basis for specifying the context in which usability should be evaluated. Laboratory evaluations of the system by personnel intimately acquainted with it are likely to produce user acceptance results which are misleading when the system is later rolled out in the training room.
Where an existing system is to be upgraded or enhanced, the context may already be well understood. There may be extensive results from user feedback, help desk reports and other data which will provide a basis for prioritizing user requirements for system modifications and changes. For new products or systems, it will be necessary to gather information about its context of use through interviews and meetings with project stakeholders.
The context in which the system will be used should be identified in terms of:
Quality of Use of Multimedia Learning Systems: Practical Considerations 185
• The Characteristics of the Intended Users. Of greatest relevance to the development of training systems is the realization that there is usually more than one type of user. The leamer, or the end user, is simply one potential user among many. Consideration should also be given to the role of:
• trainers;
• managers of the learning process;
• purchasers;
• support technical staff;
• knowledge providers;
• evaluators.
• The Tasks the Users will Perform. Clearly, the major goal of the learner is to acquire knowledge with the assistance of the system. A hierarchical breakdown of this global task is likely to be similar to the map of the learning domain to be covered. However, once the other kinds of users of the system are defined, it will be seen that they may have goals quite different from learning objectives alone, and the goals of these latter groups may well be phrased in standard task-description terminology. This kind of description should include the overall goals of use of the system for this category of user, as well as the characteristics of tasks which may influence usability in typical scenarios, e.g., the frequency and the duration of performance. The description should include the allocation of activities and operational steps between the human and technological resources. Tasks should not be described solely in terms of the functions or features provided by a product or system.
• The Environment in which the Users will Use the System. With the availability of multimedia workstations of widely differing capabilities the design team faces an unenviable task: that of designing for run-time environments which may differ radically in unpredictable ways. In addition, delivery mechanisms (the Web, LAN, CD-ROM, or a combination of all of these) make the task of settling on one kind of technical environment even more complicated. It is, nevertheless, important at this early stage to set down some markers as to what the minimal as well as the optimal system requirements should be, with the intention to user-test in these environments before release. Relevant characteristics of the physical and social environment also need to be considered. Although an office environment may well be a standard to which the design is addressed, the legislative environment (e.g., laws, ordinances, directives and standards) and the social and cultural environment (e.g., work practices, organizational structure and attitudes) have also to be considered. Different parts of Europe have widely-differing expectations of the style of presentation that is expected; much more so different continents (cf for instance the difference between the degree of formality expected in the USA and that expected in, say, Germany or Sweden).