Microsoft Word C029024e doc Reference number ISO/TS 16951 2004(E) © ISO 2004 TECHNICAL SPECIFICATION ISO/TS 16951 First edition 2004 03 15 Road vehicles — Ergonomic aspects of transport information an[.]
Appoint an examiner
The priority index procedure requires an examiner to coordinate the data preparation, analysis, and reporting
The examiner shall be familiar with the prioritization process, knowledgeable on message management, and have automotive experience
If the examiner has similar qualifications to that of the evaluators, the examiner may participate as both an examiner and evaluator.
Identify and assemble messages
The examiner must pre-identify the messages intended for drivers, which typically reflect the combined output from both TICS and non-TICS linked to a message management system These messages should be gathered and organized by the examiner for effective presentation to the evaluators.
Define driving context and situation
The examiner will identify or help establish the driving context and situation, focusing on the road environment and traffic conditions relevant to each message.
Defining contextual and situational factors at the time of message presentation is crucial, as the urgency and criticality ratings are significantly influenced by the driving context It is essential to focus on how messages are conveyed in potentially hazardous situations, taking into account all relevant hazards while excluding highly unlikely scenarios Consequently, the examiner should identify one or more scenarios—typically no more than four—that correspond to a specific message.
The same message can be interpreted as distinct information in varying scenarios due to the different driving contexts that influence message priority For instance, the urgency of a system malfunction alert will differ significantly depending on whether the driver is starting the vehicle or actively driving.
For effective communication, it is essential to utilize a single scenario that embodies a reasonable "worst-case" situation for the message being conveyed At the very least, there should be consensus between a TICS or non-TICS expert and the examiner regarding this worst-case driving scenario, and consulting a traffic safety expert may also be beneficial.
3.3.1 Consider the sensing capability of the vehicle
The messages delivered to the driver are influenced by the vehicle's ability to sense and detect important situations For instance, if the vehicle can assess the driver's level of arousal, this information can be integrated into the overall driving scenario.
For situations in which the vehicle is incapable of sensing, the message priority should be determined for a scenario representing a “reasonable” worst-case situation for the factors listed in 3.3.2
3.3.2 Factors to consider in developing the driving scenarios (see Table 1)
Table 1 serves as a valuable resource for examiners to establish driving contexts and relevant situations It includes a sample of situational and contextual factors in the rightmost column, which can be utilized to define various driving scenarios If a context does not fit any of the listed candidate factors, it should be explicitly described in the “Other” section In cases where neither the context nor the situation aligns with the candidate factors, the examiner should select “not defined (N-D)” from the available options.
The following factors should be considered in developing the driving contexts and situations for evaluators to consider when making their ratings
Copyright International Organization for Standardization
The trip context encompasses the purpose of the journey, such as commuting or leisure, the timing or location along the route, and the distance needed to prepare for the next maneuver.
EXAMPLE “Close (e.g 20 m) to turn (or merge)”
The road environment significantly influences driving behavior, encompassing factors such as road type (highway, urban, or country road), speed limits, lane numbers, and road width Additionally, the surrounding environment, including weather conditions and the time of day (morning, daytime, nighttime, or rainy conditions), plays a crucial role in driving safety and performance.
Traffic conditions are constantly evolving and are influenced by various factors, including road obstacles and the behavior of other vehicles Key considerations involve maintaining appropriate headway distance and managing speed differences relative to the vehicle in front.
EXAMPLE 3 “Speed difference to the lead vehicle”
This factor originates from the vehicle itself and derives from the relationship between the vehicle and the road Vehicle condition can be separated into “vehicle type” and “vehicle state”
EXAMPLE 1 Vehicle type: “passenger vehicle”, “heavy vehicle”
EXAMPLE 2 Vehicle state: “driving speed”; “driving in left [right] lane”; “negotiating curve [intersection]”; position of the vehicle within a lane
3.3.3 Document the driving context and situation
The defined driving context, situation, appropriate driver behaviours and/or cognitive demands associated with each message shall be documented An example for a specific configuration is given in Annex E
Table 1 — Factors to consider in developing driving scenarios Driving context/situation Candidate situational or contextual factors
Close to turn (or merge, or diverge) Other (……)
Highway / urban / country / curve / icy / wet Speed limit (…)
Rain / fog Time of day (morning / daytime / night-time) Other (……)
Headway distance is approx (…) m or (…) s Lateral vehicle(s) exists (yes or no)
Speed difference to the lead vehicle is approx (…) km / h N-D
Passenger vehicle / heavy vehicle Other (……)
Driving speed (…) km/h (mph) Driving in left (or right) lane Curve (intersection) negotiation Other (……)
Systems check after start-up shows all systems are normal Malfunction of system (……) — provide relevant details Other (……)
Miscellaneous Driver’s state of arousal
Copyright International Organization for Standardization
Select the evaluators
The examiner shall select a minimum of 5 evaluators (see 4.1 and Annex C)
Evaluators should consist of seasoned human factors and road safety experts who have a comprehensive understanding of the TICS system's functions They need to be knowledgeable about the traffic and road conditions specific to the country or region in question, ensuring they can assess the safe delivery of messages effectively It is also advisable for all evaluators to have practical experience with the system in use.
It is essential to document the profile of each evaluator, which should encompass their area of expertise, understanding of road safety, awareness of human factors, and familiarity with the systems that generate prioritized messages (refer to Annex D).
Evaluate criticality and urgency of a message
The examiner will clarify the message contents and driving scenarios to ensure that evaluators share a common understanding of the examined scenarios, system functions, and message details.
Examiners must ensure that evaluators grasp the definitions of criticality and urgency, which are the two key criteria for the priority index To achieve this, evaluators need to be familiar with the four-category ordinal evaluation scales utilized for assigning values of criticality and urgency (refer to Tables 2 and 3).
Each evaluator shall assign a criticality and urgency rating for each message assuming he/she is the driver
Urgency is primarily influenced by time and the controllability of a situation When a situation is uncontrollable, drivers are unlikely to take action Conversely, if there is potential for control, urgency is assessed based on the expected timing for drivers to respond and manage the situation effectively.
Evaluators should consider all hazardous situations, excluding highly unlikely scenarios, if the examiner has not clearly defined the driving context They must assume a reasonable worst-case scenario when assessing the criticality and urgency of these situations.
Instructions for the examiner
The examiner's responsibilities include recording evaluator information in the evaluator profile, creating and distributing a tailored questionnaire for the vehicle and system evaluation, and providing definitions of criticality and urgency They must clarify evaluation items and questionnaire content, allowing evaluators to document the driving context for their assessments Additionally, the examiner explains the classification of criticality and urgency, guides evaluators on how to record their ratings, collects the completed questionnaires, and analyzes the data to report the results.
Table 2 — Criticality rating scale Rating
Risk to vehicle, occupants and/or pedestrians
Ignoring speed warning when driving significantly above the speed limit
Collision as a result of loss of braking due to ignoring the brake failure warning Departing roadway due to ignoring lane departure warning
Leaving the roadway, head-on collision and collision with structures at intermediate speed
Following vehicle ahead too closely at high speed
Risk of collision due to following a vehicle ahead too closely at intermediate speed
Vehicle-to-vehicle collisions often occur at intermediate or low speeds when drivers ignore collision warnings Additionally, vehicles may leave the road, resulting in head-on collisions or crashes with structures, also at these lower speeds.
Vehicle-to-vehicle collision except head-on collision at low speed
Following vehicle ahead too closely at low speed
Collision with structures at low speed
0 No injury (no vehicle damage)
Vehicle-to-vehicle contact at very low speed
Collision with structures at very low speed
NOTE 1 Initially, three ratings scales were used in calculating a priority index See Annex F for the reason for reducing the number of rating scales to two
Only a limited number of examples are provided, and their significance may differ across various regions due to the unique road environments and contextual factors Consequently, examiners have the discretion to modify these examples as needed.
NOTE 3 The examples shown here illustrate that a warning message can be assigned a high criticality rating, even if failure to respond to the warning does not result in a crash.
Driving speeds differ across countries, with classifications of "very low," "low," "intermediate," and "high" speeds varying based on local regulations and specific driving conditions, including urban, suburban, and expressway environments.
Take immediate action or decision (within zero to three seconds) according to the displayed information
Obstacle immediately in the vehicle path Brake immediately Steer to avoid dangerous situations ACC malfunctioning
Take action or decision according to the information within 3 to 10 seconds [1]
Obstacle within a few seconds in the vehicle path Brake in a few seconds Steer away from danger as required
Prepare to take action or decision according to the information within 10 seconds to 2 minutes Onset of detection of an obstacle
No direct action or decision required by driver System on
Copyright International Organization for Standardization
Alternative method for determining message priority
The priority matrix method, outlined in Annex A, offers an alternative approach for assessing message priority This technique relies on subject matter experts conducting pair-wise comparisons of all messages, rather than utilizing the criticality and urgency ratings from Tables 2 and 3 or calculating a priority index as described in Clause 4.
4 Data analysis for priority index
General
This clause provides a method for calculating the message priority index based on evaluator ratings of criticality (c i ) and urgency (u i )
To ensure the reliability of the priority index for each message, it is recommended to have ten or more evaluators, as the average of their priority indexes influences the overall index However, fewer than ten evaluators may be acceptable if the standard deviation among them is sufficiently small An example procedure for determining acceptable standard deviations can be found in Annex I Regardless of the circumstances, a minimum of five evaluators is necessary to prevent bias in the selection process.
Select weights
To effectively utilize this Technical Specification, designers of the message management system must first determine the numerical values of \( k_c \) and \( k_u \) In the absence of established guidelines, one approach is to assign both \( k_c \) and \( k_u \) a value of 1.0, treating criticality and urgency as equally significant Alternatively, these values can be derived from empirical data, as illustrated in Annex G, which demonstrates that \( k_u = k_c = 1 \) can serve as rounded weight factors The evaluator's ratings of criticality (\( c_i \)) and urgency (\( u_i \)), referenced in Clause 3, are essential for calculating the priority index \( P_j \) in section 4.4, as shown in Equation (2) With ratings ranging from 0 to 3 for both criticality and urgency (see Tables 2 and 3), setting \( k_c = k_u = 1 \) ensures that both factors contribute equally to the priority index.
Calculate priority p ij
The priority index \( p_{ij} \) reflects the relative importance of the \( i \)th evaluator and the \( j \)th message, calculated using the formula \( p_{ij} = k_c c_{ij} + k_u u_{ij} \) In this equation, \( c_{ij} \) and \( u_{ij} \) represent the individual scores for criticality and urgency, respectively, while \( k_c \) and \( k_u \) are the weight factors assigned to criticality and urgency in relation to \( p_i \).
Calculate arithmetic mean and standard deviation of priority index across evaluators for
The arithmetic mean P j , given by Equation (2), is the priority index, which indicates the relative importance of the jth message:
P j is the priority index of the jth message;
C j , U j are the mean score across evaluators, respectively, of criticality and urgency of the jth message; n is the number of evaluators
Standard deviation σ j , given by Equation (3), indicates the relative confidence of the priority index of the jth message
1 1 n j j ij j i n n ij ij i i n n c ij u ij c ij u ij i i
(3) where σ j is the standard deviation of the jth message;
V j is the variance of the priority index of the jth message.
Calculate P j and σ j
Repeat Equations (2) and (3), calculating P j and σ j for each of the j messages.
Determine priority order
To prioritize the messages, first rank them based on their P j values in descending order, with larger values indicating higher priority In cases where multiple messages share the same P j value, the message with the higher mean criticality score C i should be given precedence.
Prepare a priority order table that lists the numerical values C j and U j , and P j and σ j , for each message evaluated
An empty table for displaying an example of a priority index calculated by assigning 1,0 to k u and k c is shown in Annex H
Copyright International Organization for Standardization
Evaluate data quality
To ensure high data quality in evaluators' priority ratings of each message, it is essential to verify a sufficient level of agreement among evaluators This can be achieved through two methods: either more than half of the evaluators must agree on the criticality and urgency ratings of a message, or the standard deviation of the priority index must be less than 1.0 (refer to Annex I).
If the initial evaluation is unclear, examiners will interview the evaluators to determine if the message context was ambiguous In such cases, examiners will provide further clarification or additional details about the driving context and situation, prompting the evaluators to re-assess the messages.
In instances of re-evaluation, messages that initially received an average criticality score below 0.5 and an average urgency score below 1.5 will not be included in the re-evaluation process.
Prioritization of priority ranking
Designers should prioritize message rankings to prevent the simultaneous presentation of auditory messages in vehicles When multiple messages need to be displayed, those with higher priority should be emphasized Once the messages are presented, drivers should have the autonomy to select, deactivate, and cancel messages, regardless of their priority, except for those mandated by law.
Even when the priority index determination process follows Clauses 3 to 5 correctly, significant discrepancies in standard deviation among messages can occur This may arise from varying interpretations of message criticality and urgency by evaluators Consequently, it is essential to address these messages carefully, potentially by implementing specialized methods for their presentation.
How to deal with additional messages
When developing new TICS systems or messages, it is essential to prioritize these new messages Additionally, if an evaluator is replaced, the new evaluator should come from a similar discipline as the previous one.
Procedures 3.2 to 4.5 are to be executed solely when new messages are introduced The results from these procedures will be incorporated into the existing priority order table established in section 4.6 to ascertain the priority of the new messages.
Documentation
The examiner will provide a report that includes the evaluators' profiles, a list of messages related to driving situations along with the consequences of ignoring these messages, and a priority index for each message.
Other
The installed priority ranking should be communicated to the driver by appropriate means (e.g operator’s © ISO 2004 – All rights reserved 11
informative) “Acceptable” standard deviations σ j for priority indexes
The priority matrix offers an effective alternative for prioritizing competing TICS and non-TICS messages, serving as a substitute for the methods outlined in Clauses 3, 4, and 5 This approach eliminates the need for complex mathematical formulations based on criticality and urgency, instead presenting all messages in a matrix format with n columns and rows Each message is articulated as it would appear in the vehicle, clearly detailing the system's behavior or malfunctions Experts systematically compare each pair of messages, recording the one with higher priority, and this process continues until all message pairs have been evaluated.
A.2 Advantages and disadvantages of method
This method has been chosen because it fits well into the engineering process of the human-machine interface (HMI) development a) Main advantages
The method can be executed by current experts familiar with the respective systems, eliminating the necessity to engage individuals with extensive knowledge of all vehicle warning messages, which can be challenging to locate even within automotive companies.
Optimizing system reactions is straightforward, as adjustments can be made within individual cells without affecting other components of the system However, there are notable disadvantages to consider.
All possible pairs of messages should be evaluated, which can result in a large number of evaluations
System reactions based on the priority matrix require more device memory space to implement
A.3 Requirements for examiner and evaluators
The examiner must possess expertise in the evaluated TICS systems and have experience with HMI, or be an HMI expert with a solid understanding of TICS systems Additionally, the examiner should be well-versed in the prioritization process, knowledgeable about message management, and have a background in the automotive industry The responsibilities of the examiner are outlined in section A.5.
Evaluators will be chosen from specialists in both TICS and non-TICS systems, with at least one expert from each TICS required Additionally, HMI experts may also participate as evaluators.
An expert evaluator must have a strong grasp of the functions of both TICS and non-TICS systems, as well as a comprehensive understanding of the traffic and road environments specific to the country or region in question.
Copyright International Organization for Standardization
ISO/TS 16951:2004(E) have the ability to evaluate and take into account the safe presentation of messages Actual use of the system is recommended for all expert evaluators
Each expert evaluator's profile must be documented, encompassing details such as age, gender, area of expertise, years of experience in their respective fields, and duration of experience specifically related to TICS human interfaces (refer to Annex D).
The expert evaluator prioritizes each message based on their comprehensive understanding of the system and the various messages available for display.
When prioritizing messages, it is essential to consider both criticality and urgency, as these factors are inherently part of every priority decision Unlike the priority index method, there is no requirement to explicitly assess criticality and urgency to establish the appropriate priority level.
The examiner is responsible for gathering messages intended for drivers, which reflect the combined output from both TICS and non-TICS systems linked to the message management system Additionally, the examiner must create a priority matrix for expert evaluators, where all displayable messages are organized in a matrix format, with each message represented in individual columns and rows (refer to Table A.1).
A.4.2 Define driving context and situation
The examiner must establish a driving context and situation based on the road environment and traffic conditions relevant to each message This definition is crucial as it clarifies the driving context and differentiates between various situations, which can affect the priority of the message and the system's response.
Driving context and situation can be classified into “trip context”, “road environment”, “traffic situation”, and
When presenting messages related to "vehicle condition," it is crucial to define contextual and situational factors at the time of delivery, as their priority is significantly influenced by the driving context Special emphasis must be placed on how messages are conveyed in potentially hazardous situations, taking into account all relevant hazards while excluding highly unlikely scenarios Consequently, the examiner may establish one or more scenarios—typically no more than four—for each specific message.
For effective communication, it is essential to utilize a single scenario that embodies a reasonable "worst-case" situation for the message being conveyed At the very least, there should be consensus between a TICS or non-TICS expert and the examiner regarding this worst-case driving scenario, and consulting a traffic safety expert may also be beneficial.
Messages in driving contexts vary in priority based on the situation For instance, a message about a system malfunction holds different importance when the driver is starting the car compared to when they are actively driving This necessitates that in-vehicle sensors can differentiate between these scenarios.
In such cases, identical messages presented in different contexts should be treated as distinct information items for assessment Each context will occupy its own row and column within the priority matrix.
A.4.2.3 Document the driving context and situation