Safer Surgery part 21 docx

Integrating Non-Technical Skills into Anaesthetists’ Workplace-based Assessment Tools Ronnie Glavin and Rona Patey introduction In this chapter, the authors will provide a brief account

This page has been left blank intentionally

Integrating Non-Technical Skills into Anaesthetists’ Workplace-based Assessment

Tools Ronnie Glavin and Rona Patey

introduction

In this chapter, the authors will provide a brief account of the development of the Anaesthetists’ Non-Technical Skills (ANTS) system and then consider how this

system may find a suitable home in the UK anaesthetic curriculum (Anaesthetists’

Non-Technical Skills System Handbook).

But first, what are non-technical skills (NTS)? Within the context of anaesthesia they have been defined as behaviours in the operating theatre environment, which are not directly related to the use of medical expertise, drugs or equipment (Fletcher

et al 2003) They encompass both interpersonal skills (e.g., communication, team working and leadership) and also cognitive skills (e.g., situation awareness and

decision making) (Anaesthetists’ Non-Technical Skills System Handbook) NTS

are important because we need them to function effectively in our professional lives As with many important aspects of living, we really only notice them when they are absent or poorly performed Then, our practice is not so effective and may become dangerous In anaesthesia, and many other professions, studies of adverse incidents consistently suggest that the underlying factors are more likely

to be due to difficulties with the application of NTS rather than a lack of basic knowledge or practical skill (Flin et al 2008) Is it then reasonable to ask: ‘If NTS are so important why have they not featured prominently in anaesthetic training?’ One might reply that they might have done had we known what they were exactly

in the context of anaesthesia Until now NTS have been elusive and difficult to describe or pin down This inability to accurately define or describe NTS has resulted in them generally only being considered under a broad umbrella category

of ‘soft skills’ However, there is nothing soft about these skills The platform they provide, along with the knowledge and skills from more conventional areas of the curriculum, is the solid base required for the development of anaesthetic practice

So far, we have referred to NTS as if they were a generic set of skills found across many circumstances and professions Bob Helmreich, emeritus psychologist

at University of Texas, Austin, played a key role in the early NASA work in this field of key human factors and then oversaw the development of the concept from

Safer Surgery

176

infancy to maturity in other industries He reminds us that there is no generic set

of NTS and has warned that different organizations/professions and cultures have their own variations of NTS (Helmreich 2000) Therefore, whilst NTS identified

in other professions can give us a flavour of those used in anaesthesia, we cannot just assume that they cross directly into our practice With this in mind, in 1999, having confirmed that such work was not already under way, a research team comprising clinical anaesthetists (including both authors) and psychologists set about the business of identifying the non-technical skills important for anaesthetic practice (funded by NHS Education for Scotland)

Development of the Anaesthetists’ non-Technical Skills System

How did we do this? We believed that anaesthetists had always learned and used NTS but that this was largely an unconscious process Therefore, we could not simply ask anaesthetists ‘What non-technical skills are important in your practice?’ Nor would the quantitative or positivistic approach, which was the research tradition that the clinical anaesthetists in the team were familiar with, be appropriate This was not an investigation where quantitative measurement, such as where a new technique is developed to impact on blood pressure or cardiac output, would help

A different approach was required A set of task analysis techniques was used with questionnaires, observations and interviews For the last, a form of cognitive task analysis was employed (Crandall et al 2006, Fletcher et al 2004) This is a qualitative research method whereby the researcher uses a set of probes, usually in the form of questions, to help the interviewee (in this case consultant anaesthetists) describe one

or more episodes of clinical practice The interviews contained the raw data and were analysed by extracting all the behaviours and skills that were mentioned These approaches allowed access to behaviours which may not otherwise have impinged upon the consciousness of the participants These elicited behaviours do not come neatly ordered, and so for clarity and ease of use, a system has to be assembled from the data For this, a series of iterative steps took place where the research team developed a prototype categorization of the principal behaviours and took rudimentary versions of the system into the workplace and studied their apparent effectiveness After several iterations the current Anaesthetists’ Non-Technical Skills (ANTS) system was arrived at (see Table 11.1)

The next step was to evaluate some key properties of this system Was it valid? Could it be used in a reliable manner – would different assessors give similar scores to the same performance? At an even more fundamental level, could clinical anaesthetists actually use the system? To address these questions, 50 consultant anaesthetists across Scotland took part in a study They attended a one-day session comprising a morning of teaching and familiarization with the concepts behind the system and the system itself followed by an afternoon of applying the system

to eight video-recorded anaesthetic scenarios that had been conceived, performed and produced by the research team The findings have been published elsewhere

(Fletcher et al 2003), but the main conclusions were that the clinical anaesthetists could use the system and could provide broad consensus on the scores awarded, and that the system captured the NTS observed However, these anaesthetists were scoring performances while observing them on videotape, without any other distractions It was important to assess if the system could be used in the workplace Here, if trainers were to apply the system to assess the performance of

a trainee in order to provide feedback, they would also be required to continue to monitor the patient and the operating theatre environment

Of the original volunteers, 17 participated in the usability study The consultant anaesthetists and the trainees being supervised were asked to complete questionnaires

at the end of the theatre session The results of this usability study, which are detailed elsewhere, were positive (Patey et al 2005) It seemed that the system could be used in the workplace and furthermore participating trainers and trainees responded in a very positive manner to use of the system as a way of providing formative assessment and teaching of NTS Was the anaesthetic community in the

UK now ready to adopt the ANTS system into routine training?

early experiences of Promoting the AnTS System

The body responsible for standards of anaesthetic practice and training of anaesthetists in the UK is the Royal College of Anaesthetists Presentations on the ANTS system to the relevant committees and bodies of stakeholders produced a very positive response Following this, the next hurdle was how to best assimilate

Task management Planning and preparingPrioritizing

Providing and maintaining standards Identifying and utilizing resources

Team working Coordinating activities with team membersExchanging information

Using authority and assertiveness Assessing capabilities

Supporting others

Situation awareness Gathering informationRecognizing and understanding

Anticipating

Decision-making Identifying optionsBalancing risks and selecting options

Re-evaluating

Table 11.1 The AnTS system: categories and elements

Safer Surgery

178

the ANTS system into the teaching and assessment of anaesthetists The usability study had shown that individual consultant anaesthetists could use the system, but could the ANTS system be incorporated into the teaching practice of a whole hospital department of anaesthesia?

The proposed strategy for introduction to a department was to use a cascade system for training the trainers One of the authors of this chapter would visit several UK anaesthetic departments that had volunteered to trial the use of the ANTS system and deliver a one-day teaching session for local enthusiasts The cascade approach was based on the notion that those who had attended the one-day course would use the system, become more familiar with its application and then reach a stage where they would be comfortable to instruct their consultant colleagues Similar to the initial system evaluation study, the training day would consist of a morning going over the background to NTS, followed by familiarization with the ANTS system In the afternoon, the performance of anaesthetists in video-recorded scripted scenarios would be assessed allowing practice using the system The afternoon differed from the research work in that the participants were encouraged to discuss their scores to allow calibration to take place When the participants reached consensus and awarded very similar scores then that part

of the session ended Would this cascade approach work?

Although NTS could be identified and discussed by consultants during ANTS training sessions, on the whole the cascade approach was not successful It seems from discussion with consultants that the reasons for this were multifactorial (Dodd 2005) Firstly, most consultants only had limited familiarity with NTS and experience with the ANTS taxonomy (only the single day of training and some recommended reading) and wanted further opportunities and support to become more familiar with the tool before using it in the workplace Accurately observing behaviours is a skill, like any other, which requires deliberate practice, and although consultants regularly participated in clinical training they had not received training in behavioural observation Where the consultants had no formal training or experience in providing structured feedback (which was often the case), they were particularly uncomfortable In all probability, too many new changes were introduced at one time but it is worth exploring the intended training approach and barriers to its success in greater detail

A conventional approach to instructing anaesthetic trainees in the use of the NTS highlighted in the ANTS system would be to incorporate the underlying theoretical concepts into classroom-style teaching and then build on that base during clinical training by demonstrating application of those principles Knowledge and practice would then be formally assessed in some kind of examination We were adopting

an entirely different approach, where the system was being used to score the performance of a trainee anaesthetist in the workplace The scores awarded for the relevant categories and elements were intended to form the basis for a discussion between consultant and trainee as a way of drawing attention to the trainees’ existing strengths and limitations in the use of NTS and then plan for further development We had made the assumption that consultant anaesthetists were used to assessing trainees

on accompanied theatre lists and then providing structured feedback However, although consultants had considerable experience assessing the performance of trainees this was done in a very informal manner Many consultants in the UK had little experience in the use of formal workplace-based assessment tools This was not surprising because no such tools were in widespread use in the UK

Workplace-Based Assessment and the Curriculum in Anaesthesia

One result of this limited use of workplace-based assessment tools is that little attention has previously been paid to the requirements to support consultants in providing structured feedback in the workplace Consultant training, to highlight the essential role of feedback in both optimal learning and in the use of effective feedback techniques, has been limited In addition, it can be difficult during a busy working day for the consultant and trainee to identify time for feedback If time can be identified, perhaps at the end of the day, it may not be close to the event under consideration, and at this time both parties may be concerned with other duties such as post-operative or pre-operative visiting The conditions during a training session in operating theatres are also not ideal The presence of the whole theatre team can be inhibiting, particularly where there is discussion to be had on

an area where there are difficulties The authors have noted that this seems to cause particular concern where NTS are concerned and consultants perceive that they are commenting on personality-related matters These various factors all conspire

to make it easy to delay a feedback discussion

When time can be identified for feedback, use of the ANTS system should actually reduce the anxiety of commenting on a trainee’s personality traits The system allows discussion and emphasis on specific NTS behaviours either observed or thought

to be deficient or absent, rather than comments which are difficult to interpret or may be interpreted as personality assessments Discussion can then follow on how these specific behaviours impact on the ANTS system elements and categories The consultant can then encourage the trainee to consider why these behaviours were observed or not However, as we have previously noted, providing feedback is not the only skill in which consultant anaesthetists lack training Accurately observing behaviour is a skill in itself and requires deliberate practice for learning

This situation, where there is limited use of formal workplace-based assessment,

is changing in the UK in response to the processes brought about under the umbrella

of Modernising Medical Careers (The Foundation Programme) The body with regulatory powers for postgraduate medical training (the Postgraduate Medical and Training Board, referred to as PMETB) with the right to approve curricula, including assessment tools, is promoting workplace-based assessment tools as one method for providing evidence that trainees are achieving the appropriate competencies for their respective curricula (The Postgraduate Medical Education and Training Board) The proposed workplace-based tools are being modified from tools introduced in the UK in 2005 for use during the first two years of postgraduate training (the UK

Safer Surgery

180

Foundation Training programme) These tools are the Mini Clinical Evaluation Exercise (MiniCEX), the Directly Observed Procedural Skills (DOPS), the Case Based Discussion (CBD) and a form of 360° feedback The MiniCEX can deal with many aspects of interaction with a patient including history taking, conducting

an examination or explaining benefits and risks of procedures The DOPS focuses

on procedural skills including the decision-making, preparation and set up of the procedure The CBD is intended to look at decision-making processes by referring

to information recorded by trainees in the patient’s case notes, while the 360° allows peers and other colleagues from a range of professions to comment on professional attitudes and teamworking skills Specially modified versions of these tools are now described for anaesthetic speciality training (The Royal College of Anaesthetists,

Workplace-based Assessment).

At the time of writing, there is a sense of concern in the anaesthetic community with regard to these tools Consultant anaesthetists have embraced the principles behind workplace-based assessment However, as we found when considering use

of the ANTS system, consultants have expressed concern about the level of their training, support and preparation for using these tools Moreover, for many who have not yet considered NTS, these reactions have been focused on areas of anaesthetic practice which have been traditionally assessed (technical skills) It is therefore hardly surprising that asking consultants to conduct workplace-based assessments

of NTS, an area in which they had no previous experience, was a step too far It is also likely that the cascade approach was overambitious in that too much was asked

of too few people without ongoing support (Grant and Stanton 1999) Nevertheless, despite the cascade approach having had limited success, the Royal College of Anaesthetists has continued to support the concept of using the ANTS system and it featured as one of the assessment tools submitted to PMETB for approval

using the AnTS System to Frame Teaching

However, the difficulty with the cascade approach to incorporating NTS into anaesthetic training was not just a problem of becoming familiar with observing behaviours, using the ANTS system and considering how best to give feedback There were barriers because of the very nature of clinical training in the UK In this training system the trainee spends regular time providing anaesthetic care with a trainer, gradually taking a more central role in the clinical work in a wider range of clinical settings This would seem to be ideal in that there are regular opportunities for the trainer to observe the trainee’s practice and provide feedback The method

of use of the ANTS system which was favoured by the 17 anaesthetists during the usability study was to allocate the perioperative management of a particular patient

to the trainee accompanying them on the theatre list They would then assess the performance of the trainee and, using the observed behaviours, provide feedback after the case The consultant, of course, retained overall responsibility and could step in and make changes if necessary but, in the absence of concerns about patient

or trainee safety, would let the trainee handle the management of the patient This approach had some implications The trainer does not only have the trainee to observe but also the clinical work As a senior trainee has more experience than

a junior trainee, the consultant is more likely to have confidence that the trainee will identify the key issues involved with the management of a wider range of patients for either the whole anaesthetic management or specific parts of a case There are therefore more opportunities to delegate responsibility to a senior trainee than a junior trainee Consultant participants in the study reported that they were better able to use the system with senior rather than junior trainees There is some apparent logic in this finding If we want to look at ANTS categories such

as decision-making then it is necessary to let trainees make decisions, therefore you have to delegate that responsibility This message was also conveyed when evaluating the cascade training

One might therefore imagine that the system could be used satisfactorily with senior trainees at least However, frequently senior trainees are given responsibility not only for individual patients but for a complete surgical list, either intentionally

to provide them with required clinical experience, or to cover for leave or sickness Although a consultant anaesthetist will have supervisory responsibility for that trainee, they may be occupied elsewhere and unable to give continuous personal supervision These factors limit the operating lists where a consultant and a senior trainee work together, and participants in the cascade training initiative listed this

as another factor contributing to their difficulties in using the ANTS system In addition, often, no sooner had a trained consultant explained the system than the trainee was re-allocated to another theatre list

Given the various difficulties which had been encountered with the cascade approach, it was important to consider whether there were further approaches which could be used in addition As the clinical anaesthetists in the research team continued to use the system in their own practice, some other uses began to become apparent It was possible to use the system as a framework on which to base both the NTS elements and also the more conventional technical tasks during anaesthesia The reluctance of consultants to use the ANTS system with junior trainees was partly based on a concern that anaesthetists in the early stages of training have large areas of knowledge and practical skills to assimilate and struggle to cope with an additional load of considering NTS However, if the teacher or instructor was using the ANTS system as a framework or mental model then he or she need not communicate that framework to the learner The consultant as instructor could use the system to provide scaffolding for the trainee without this being visible to the trainee Some examples will help to clarify this process

Let us begin with the category of ‘Situation Awareness’ One of the elements

in the ANTS system category Situation Awareness is gathering information A

key technical skill to be mastered by novice anaesthetists is taking an anaesthetic history from a patient This skill resembles the history taking skills that medical undergraduates have to master but there is sufficient difference to require specific additional instruction When not familiar with the ANTS system, the trainee will

Safer Surgery

182

view this skill as history taking, not as gathering information Of course there is more to gathering information for situation awareness than history taking from the patient Information about the intended surgical procedure is required to formulate a suitable anaesthetic plan Trainees are traditionally taught to ask for the information about the proposed surgical procedure that they require, such as how long will it take; what position will the patient be in; does the surgeon need access

to the abdominal, chest or pelvic cavities; what is the anticipated blood loss? This information may be gathered from a variety of sources such as the surgeons and

the operating department nursing staff The trainee is now exchanging information and coordinating activities with team members, which are elements from the

category Team Working

As a further example, during the course of anaesthesia there will be episodes where the anaesthetic trainee needs to focus on surgical events, such as when the surgeon makes the skin incision At this time the anaesthetist should be observing the response of the patient to confirm that the depth of anaesthesia is appropriate

for that patient The relevant elements are gathering information and anticipation from Situation Awareness, prioritizing from Task Management and exchanging

information from Team Working Prioritization is included because the anaesthetist

should not be involved in other activities that are less essential (such as drawing up antibiotic drugs) The anaesthetist should anticipate the skin incision by observing the progress of the prepping of the surgical field and the placing of sterile drapes around the surgical field They should be alert for an exchange of information (a surgeon with appropriate NTS will ask if it is all right to perform the incision and the anaesthetist should be in a position to reply)

Taking this method further, the relevant NTS behaviours can be discussed during procedural skills training and the key role of NTS for safety can be highlighted

To illustrate this technique, consider the insertion of a central venous line The trainee is taught to attach cardiac monitoring so that they can be aware of arrhythmia

generation should the guide wire be passed too far Of course this allows information

gathering to provide more accurate Situation Awareness of how the technical

procedure is proceeding There can be discussion of where the monitor is best placed

for information gathering As this is sometimes found to be out of clear sight for the

person inserting the cannula, the trainee may suggest that they would ask an assistant

(exchanging information) to watch the monitor during the insertion (coordinating

activities) It is not uncommon for trainee anaesthetists to assume that their assistants

in theatre are familiar with all their activities In this situation the trainee may assume that the assistant will understand both what to look for and when to look

at the monitor In fact this is often not so, and facilitating discussion with ANTS elements and categories clarifies the importance of the exchange of information and coordination of activities so that the trainee and his/her assistant have shared situation awareness and the safety of the procedure for the patient will be enhanced

At this time, in the earliest stages of postgraduate training, it is likely that most UK medical graduates will not be aware of the concept of NTS The novice trainee may not see the behaviours being highlighted in the examples given above

as NTS They may just be seen as behaviours to be copied and applied as part of the initiation into the profession of anaesthesia However, where the instructor carries the framework and uses it to structure the components of practice, this can help direct the behaviours of the novice into good habits of practice Then, when the trainee is introduced to the theoretical aspects of NTS, there are examples of clinical practice readily available to be used for illustrative purposes Furthermore, when trainees have familiarity with the ANTS system, they can be asked to incorporate NTS when they articulate discussion of cases or events

To summarize, we can say that a trainer who is familiar with the ANTS system can use that system as a framework to help trainees acquire non-technical skills from the earliest stages of training It will also have become obvious that there can

be considerable overlap between anaesthetists’ technical and non-technical skills

A trainee anaesthetist can only anticipate what will happen during an operation if they have acquired some experience of that procedure In a similar vein a trainee can only anticipate what impact a particular drug will have on a patient if that trainee has the necessary understanding of physiology and pharmacology It may now become a little clearer why in the early days of applying the ANTS system the emphasis was on use with more experienced trainees

Future Directions

So, where do we go from here? Let us review the position briefly

The ANTS system can be used in the clinical workplace

The cascade model introduced by the Royal College of Anaesthetists to introduce the ANTS system as a tool to be used at departmental level has not been successful

Workplace-based assessment tools are now being introduced into postgraduate medical training in the UK

The proposed assessment tools have been adapted from existing tools and are being modified for purpose

There appears to be an overlap between technical and non-technical skills

in anaesthesia

One possible strategy is to incorporate appropriate aspects of the ANTS system into the miniCEX, DOPS and CBD workplace-based assessment tools As anaesthetists learn to use these tools in the workplace they can begin to include NTS in the manner outlined previously In this way, we may assist consultant anaesthetists in becoming more familiar with the ANTS system An experimental MiniCEX form (Figure 11.1 and 11.2) was designed in a collaborative venture between UK and New Zealand departments Results from the UK limb are in the process of being collected and analysis is awaited

1

2

3

4

5