Safer Surgery part 40 pdf

The aim of the present chapter was to discuss the characteristics of surgical decision-making, highlight how some behavioural and naturalistic decision research approaches can be applied

ratings were not significantly different from trainee surgeons’ self-ratings (t(37) = 0.88, p > 0.05), thus indicating agreement in the assessment of skill.

In the second series of simulations, decision-making was rated significantly

lower than all other skills (all ps < 0.05) Moreover, there was a significant pre-post

training improvement in the ratings of decision-making for the surgical trainees

(M Pre-training = 2.51 vs M Post-training = 3.62; F(1, 15) = 6.59, p = 0.05) Taken together,

these studies suggest that simulation-based team training is feasible and has great potential as a training tool From the decision-making perspective, simulations offer a unique environment, in which the skill can be observed and assessed ‘in action’ – in other words, in a dynamic, fluid, potentially stressful, but safe training environment

(v) Concluding Remarks

Surgical decision-making is an important but under-researched field The aim of the present chapter was to discuss the characteristics of surgical decision-making, highlight how some behavioural and naturalistic decision research approaches can

be applied to surgery and, finally, present empirical work that has been carried out

by our research team to date to assess making processes and decision-making as a skill in surgery We presented empirical applications of three different approaches to surgical decision-making – namely, knowledge elicitation from experts, experimentation and modelling using Judgement Analysis (JA), and, finally, simulation-based assessment of and training in decision-making (among other non-technical skills)

All approaches yielded promising findings Knowledge elicitation resulted in the representation of the care pathway for patients presenting with symptomatic gallstone disease – both acute and non-acute The approach was shown to be reliable Its contribution is that it can be used to articulate key clinical decisions that need to be made in the process of care of such patients and, importantly, what are the cues (information or considerations) that feed into these decisions Experimentation and modelling using JA is a different approach, especially useful where cognitive processes that are not easy to articulate consciously are involved In the presence of gold standard models of the type that we used in our study, JA can be used to assess surgeons’ risk estimation and to provide individualized

Mean nOTeChS subscale rating (standard deviation)

Communication vigilance Teamworking Leadership

Decision-making

Trainer 4.00 (0.97) 4.11 (1.17) 3.96 (1.15) 3.78 (1.01) 3.95 (1.10) Trainee 3.69 (1.03) 3.67 (1.15) 3.76 (0.91) 3.73 (0.90) 3.74 (0.94)

Table 21.1 Non-technical skills in the first simulation series

feedback regarding (in)appropriate cue utilization Although accurate judgement

of the likelihood of conversion of laparoscopic cholecystectomy to the open procedure is not a critical life-or-death issue, it is important in terms of informed consent It may also be thought of as an ‘exemplar judgement’ – thus suggesting that findings from this study may be generalizable across other similar surgical risk judgements Importantly, the technique does not rely on self-report

Finally, simulation-based assessment allows the simultaneous assessment of decision-making and other skills, technical and non-technical Simulation-based training that involves an entire operating theatre team recreates the working reality

of the surgeon and can be used as a training environment for the whole surgical team Existing observational tools, after appropriate revision, were shown to be reliable

in the surgical context Given the wide-spread use and face validity of surgical simulators, this approach can be used extensively both as a training tool, but also

as a research environment Currently, we are building on our initial experiences with each one of the three approaches presented here to expand the field of their application We have used interviews to assess in detail surgeons’ perceptions

of the stressors that they face in the operating theatre (Arora et al 2009, Wetzel

et al 2006) We found that technical issues (e.g., difficult anatomy; bleeding), malfunctioning/lack of availability of equipment, distractions/interruptions and poor teamwork/communication are the key stressors that surgeons have to cope with We also found that surgeons recognize the impact of such stressors on their performance – including their intra-operative decision-making Importantly, the interviews allowed us to capture a range of training needs that the surgeons articulated in relation to a stress management training module (Arora et al 2009)

We are now in the process of developing such a module, which will involve simulation-based team training One of our key aims is to assess the impact of stress management training on intra-operative decision-making via observation (thus following up previous simulation-based assessment of decision-making and other non-technical skills)

We are also piloting the use of JA as a training tool We recently used JA-derived feedback to improve accuracy and reliability of surgical risk estimation (Jacklin et

al 2009) Pre-feedback, we assessed accuracy and reliability of trainee surgeons’ and medical students’ estimates of operative mortality for major surgery using a number of patient vignettes with varying risk factors Vignette construction was guided by a published risk model as a gold standard Post-feedback, participants were retested on a second, equivalent case set We found that feedback improved reliability of risk estimates in both groups and also accuracy in students’ risk estimates: these estimates were significantly worse than those of the surgical group pre-feedback, but matched them in accuracy post-feedback Accuracy of the surgeons’ estimates did not improve (arguably, because of a ceiling effect) This is

a novel application of JA, suggesting that the technique could be potentially useful for surgical training and assessment – at least in the field of risk estimation All approaches that we have used so far have limitations, both conceptual, but also practical Over-reliance on self-report is an obvious limitation of knowledge

elicitation/any interview-based technique The usefulness of JA-type modelling will always be a function of the availability of robust epidemiological models to

be used as gold standard comparisons, whilst the choice of modelling approach

to be applied to the surgeons’ judgements can be debated Simulation is a rather expensive tool: it requires facilities and trained trainers In addition, robust transfer

of learning from simulated to real operating theatres remains to be empirically demonstrated

These limitations, together with the variety of environments in which surgeons are required to make decisions and the range of potential applications of this line of research render a multimodal approach to surgical decision-making of paramount importance First of all, surgical decision-making occurs in the operating theatre under time pressure and stress, but also in surgical wards and outpatient clinics, in which the surgeon has more opportunity to consider options and discuss them with the patient Some of the decisions are grounded on highly technical knowledge and the required skill to execute them; others involve more reliance on patient’s preferences; all of them require systematic use of the available evidence base It

is rather hard, if not impossible, to assume that all the approaches that we have described in this chapter are equally applicable to all decision situations

Secondly, measuring surgeons’ decision-making implies that it is feasible and conceptually sound to treat decision-making as an observable skill, in which surgeons can be trained In turn, training involves demonstration of tangible improvement and also assessment (formative, summative, or both) Although some initial attempts can be found in the relevant literature (e.g., Sarker et al 2009), these are still few and rather heavily knowledge-based – which suggests that their generalizability across decision-making situations remains to be demonstrated Systematic, replicable empirical work across a variety of decision-making situations needs to be done in order to arrive at robust, valid tools to assess decision-making skill comprehensively Such tools are not unlikely to consist of a range of different modules – thus representing the richness and variety of decisions that a modern surgeon is faced with The multimodal approach that we propose here appears to be well suited to capture the various elements of real-life surgical decisions, thereby rendering surgical decision-making less of a ‘black box’ among other surgical skills

Acknowledgements

The work that is reported in this chapter has been funded by a research fellowship

of the Royal College of Surgeons of England (RJ), the Rosetrees Foundation (RJ), the Grand Lodge 250th Anniversary Fund (RJ), a research fellowship of the Economic and Social Research Council Centre for Economic Learning and Social Evolution (NS) and the British Academy (NS)

Authors’ note

Papers outlining methods of assessing surgical decision-making as described in this chapter were first presented at the 21st Society for Probability, Utility and Decision-Making (SPUDM) biennial research conference (Warsaw, 19–23 August, 2007) and at the 2nd Healthcare Systems, Ergonomics, and Patient Safety (HEPS) international conference (Strasbourg, 25–27 June, 2008) We would like to thank all our clinical and psychologist colleagues for their very useful feedback

References

Aggarwal, R., Moorthy, K and Darzi, A (2004) Laparoscopic skills training and

assessment British Journal of Surgery 91, 1549–58.

Arora, S., Sevdalis, N., Nestel, D., Tierney, T., Woloshynowych, M and Kneebone,

R (2009) Managing intra-operative stress: What do surgeons want from a crisis

training programme? American Journal of Surgery 197, 537–43.

Calland J., Guerlain S., Adams R., et al (2002) A systems approach to surgical

safety Surgical Endoscopy 16, 1005–14.

Canter, R and Kelly, A (2007) A new curriculum for surgical training within

the United Kingdom: The first stages of implementation Journal of Surgical Education 64, 20–6.

Chapman, G.B and Sonnenberg, F.A (2003) Decision Making in Healthcare

Cambridge: Cambridge University Press

Clarke, M.G., Wilson, J.R.M., Kennedy, K.P and MacDonald, R.P (2007) Clinical judgment analysis of the parameters used by consultant urologists in

the management of prostate cancer Journal of Urology 178, 98–102.

Coiera, E and Tombs, V (1998) Communication behaviours in a hospital setting:

An observational study British Medical Journal 316, 673–6.

Cooksey, R.W (1996a) Judgment Analysis: Theory, Methods, and Applications

San Diego, CA: Academic Press

Cooksey, R.W (1996b) The methodology of social judgement theory Thinking and Reasoning 2, 141–73.

Dankelman, J and Di Lorenzo, N (2005) Surgical training and simulation

Minimally Invasive Therapy and Allied Technologies 14, 211–13.

Denig, P and Haaijer-Ruskamp, F.M (1994) ‘Thinking aloud’ as a method of

analyzing the treatment decision of physicians European Journal of Public Health 4, 55–9.

Ericsson, K.A and Simon, H.A (1984) Protocol Analysis: Verbal Reports as Data Cambridge, MA: MIT Press

Flin, R., Martin, L., Goeters, K.M., Hörmann, H.J., Amalberti, R., Valot, C., Nijhuis, H (2003) Development of the NOTECHS (non-technical skills)

system for assessing pilots’ CRM skills Human Factors and Aerospace Safety

3, 97–119

Flin, R., Youngson, G and Yule S (2007) How do surgeons make intraoperative

decisions? Quality and Safety in Health Care 16, 235–9.

Frank, J.R (2005) The CanMEDS 2005 Physician Competency Framework Better Standards Better Physicians Better Care Ottawa: Royal College of

Physicians and Surgeons of Canada

Frank, J.R, Jabbour, M and Tugwell P., et al (1996) Skills for the new millennium:

Report of the societal needs working group, CanMEDS 2000 Project Annals Royal College of Physicians and Surgeons of Canada 29, 206–16.

Frank, J.R and Langer, B (2003) Collaboration, communication, management,

and advocacy: Teaching surgeons new skills through the CanMEDS project World Journal of Surgery 27, 972–8.

Fried, G.M and Feldman, L.S (2008) Objective assessment of technical

performance World Journal of Surgery 32, 156–60.

Harries, C and Kostopoulou, O (2005) Psychological approaches to measuring and modelling clinical decision-making In A Bowling and S Ebrahim (eds)

Handbook of Health Research Methods: Investigation, Measurement and Analysis (pp 331–61) Maidenhead: Open University Press.

Healey, A.N and Vincent, C.A (2007) The systems of surgery Theoretical Issues

in Ergonomic Science 8, 429–43.

Healey, A.N., Sevdalis, N and Vincent, C.A (2006a) Measuring intra-operative

interference from distraction and interruption observed in the operating theatre Ergonomics 49, 589–604.

Healey, A.N., Undre, S and Vincent, C.A (2006b) Defining the technical skills of

teamwork in surgery Quality and Safety in Health Care 15, 231–4.

Helmreich, R.L., Merritt, A.C and Wilhelm, J.A (1999) The evolution of Crew

Resource Management training in commercial aviation International Journal

of Aviation Psychology, 9, 19-32.

Hertzer, N.R 2006 Current status of endovascular repair of infrarenal abdominal

aortic aneurysms in the context of 50 years of conventional repair Ann N Y Acad Sci 1085, 175–86.

Jacklin, R., Sevdalis, N., Darzi, A and Vincent, C.A (2008a) Mapping surgical practice decision making: An interview study to evaluate decisions in surgical

care American Journal of Surgery 195, 689–96.

Jacklin, R., Sevdalis, N., Harries, C., Darzi, A and Vincent, C.A (2008b) Judgment analysis: A method for quantitative evaluation of trainee surgeons’ judgments

of surgical risk American Journal of Surgery 195, 193–8.

Jacklin, R., Sevdalis, N., Darzi, A and Vincent, C.A (2009) Efficacy of cognitive

feedback in improving operative risk estimation American Journal of Surgery

197, 76–81

Kee, F., McDonald, P., Kirwan, J.R et al (1998) Urgency and priority for cardiac

surgery: A clinical judgment analysis British Medical Journal 316, 925–9 Klampfer, R., Flin, R., Helmreich, R.L., et al (2001) Enhancing Performance

in High Risk Environments: Recommendations for the Use of Behavioural

Markers Report from the Behavioural Markers Workshop, Zurich, June

Berlin: Damler Benz Foundation

Klein, G (1998) Sources of Power: How People Make Decisions London: MIT

Press

Klein, G., Orasanu, J., Calderwood, R and Zsambok, C.E (1993) Decision Making in Action: Models and Methods Norwood, NJ: Ablex Publishing

Corporation

Koehler, D.J and Harvey, N (2004) Blackwell Handbook of Judgment and Decision Making Oxford: Blackwell.

Koele, P and Hoogstraten, J (1999) Determinants of dentists’ decisions to initiate

dental implant treatment: A judgment analysis Journal of Prosthetic Dentistry

81, 476–80.

Koutantji, M., McCulloch, P and Undre, S., et al (2008) Is team training in

briefings for surgical teams feasible in simulation? Cognition, Technology and Work 10, 275–85

Leong, J.J., Nicolaou, M., Emery, R.J., Darzi, A and Yang, G.Z (2007) Visual

search behaviour in skeletal radiographs: A cross-specialty study Clinical Radiology 62, 1069–77.

Lipshitz, R., Klein, G., Orasanu, J and Salas, E (2001) Taking stock of naturalistic

decision making Journal of Behavioral Decision Making 14, 331–52.

MacCormick, A.D and Parry, B.R (2006) Judgment analysis of surgeons’

prioritization of patients for elective general surgery Medical Decision Making

26, 255–64

Moorthy, K., Munz, Y and Adams S et al (2005) A human factors analysis of technical and team skills among surgical trainees during procedural simulations

in a Simulated Operating Theatre (SOT) Annals of Surgery 242, 631–9.

Moorthy, K., Munz, Y., Forrest, D., Pandey, V., Undre, S., Vincent, C and Darzi,

A (2006) Surgical crisis management skills training and assessment Annals

of Surgery 244, 139–47.

Nisbett, R.E and Wilson, T.D (1997) Telling more than we can know: Verbal

reports on mental processes Psychological Review 84, 231–57.

Pfister, M., Jakob, S., Frey, F.J., Niederer, U., Schmidt, M and Marti, H.P (1999)

Judgment analysis in clinical nephrology American Journal of Kidney Diseases

34, 569–75

Sarker, S., Chang, A., Albrani, T and Vincent, C.A (2008) Constructing

hierarchical task analysis in surgery Surgical Endoscopy 22, 107–11.

Sarker, S., Rehman, S., Ladwa, M., Chang, A and Vincent, C.A (2009) A

decision-making learning and assessment tool in laparoscopic cholecystectomy Surgical Endoscopy 23(1), 197–203.

Sevdalis, N and McCulloch, P (2006) Teaching evidence-based decision-making

Surgical Clinics of North America 86, 59–70.

Sevdalis, N and Jacklin, R (2008) Opening the ‘black box’ of surgeons’ risk

estimation: From intuition to quantitative modeling World Journal of Surgery

32, 324–5.

Sevdalis, N., Davis, R, Koutantji, M., Undre, S., Darzi, A and Vincent, C.A (2008) Reliability of a revised NOTECHS scale for use in surgical teams

American Journal of Surgery 196, 184–90.

Undre, S., Koutantji, M., Sevdalis, N., et al (2007) Multi-disciplinary crisis

simulations: The way forward for training surgical teams World Journal of Surgery 31, 1843–53.

van Avermaete J.A.G and Kruijsen E (1998) The Evaluation of Non-technical Skills of Multi-pilot Aircrew in Relation to the JAR-FCL Requirements (project

rep CR-98443) Amsterdam: NLR, Amsterdam

Vincent, C., Moorthy, K., Sarker, S.K., Chang, A and Darzi, A (2004) Systems approaches to surgical quality and safety: from concept to measurement

Annals of Surgery 239, 475–82.

Wetzel, C., Kneebone, R., Woloshynowych, M., Nestel, D., Moorthy, K., Kidd, K

and Darzi, A (2006) The effects of stress on surgical performance American Journal of Surgery 191, 5–10.

Yule, S., Flin, R., Paterson-Brown, S and Maran, N (2006) Non-technical skills

for surgeons A review of the literature Surgery 139, 140–9.

Simulator-Based Evaluation of Clinical

Guidelines in Acute Medicine

Christoph Eich, Michael Müller, Andrea Nickut and Arnd Timmermann

Clinical Background

Guidelines in Acute Medicine

Guidelines and related algorithms play a major role in acute medicine, in particular for the handling of time critical and high-risk situations in anaesthesiology, emergency and critical care medicine Well-known examples are the guidelines on cardiopulmonary resuscitation (ASA 2006, Biarent et al 2005, 2006) Furthermore, numerous national and local guidelines exist on time-critical procedures and interventions, for example, on rapid-sequence induction (RSI) of anaesthesia in children (Schmidt et al 2007) The new technique (RSI controlled) makes the potentially hazardous procedure of anaesthesia induction in a non-fasted infant less time critical It is thought to produce less stress and fewer unsafe actions and critical incidents and hence may be safer for the child

Characteristically, guidelines are based on a thoroughly performed consensus process on best scientific evidence, paired with expert knowledge and experience Though guidelines aim to combine this with clinical feasibility, infrastructural conditions and educational aspects, evidence of their actual clinical superiority, and

in particular of their whole entirety, is limited (Morley and Zaritsky 2005, Nolan 2005) A valid clinical evaluation frequently collides with ethical considerations, particularly when children are involved Hence, a high proportion of guidelines

in acute medicine may be evaluated by clinical observation, retrospective epidemiological analysis and expert appraisal only The question is whether a high-fidelity simulator in an authentic clinical environment would be able to decrease this knowledge gap

Stress and Safety

Stress is defined as a set of adaptive reactions of an individual undergoing aggression, stimulation, or effort These stimuli provoke an increase in sympathetic activity that causes a raise of heart and respiratory rate, arterial blood pressure, and metabolic processes (Vrijkotte et al 2000) Hence stress can generally be quantified by measuring cardiorespiratory and metabolic parameters

Physical and mental stress in operating room team members are both thought to have negative impact on patient safety as high stress levels are likely to be related with unsafe actions and critical events (Lazarus et al 1952, Gaba and Howard

2002, Howard et al 2003, Moorthy et al 2003, Metz 2007) As a consequence, clinical guidelines in acute medicine aim to reduce stress to increase patient safety

On the other hand, optimum stress levels which allow best individual performance are subject to interpersonal variations within a wide range (Metz 2007, Nater et

al 2007a) As there seems to be no direct or even linear correlation between stress levels and safety such as unsafe actions and critical incidents, stress measurement alone would have only limited analytical power for the evaluation of clinical guidelines In order to get more valid answers, we combine objective stress measurement with observational criteria and self-assessment (perception)

The Model

High fidelity infant simulators have only been commercially available for a few years There now exists a sizable body of experience with their use for training and assessment purposes (Gaba et al 1998, Howard et al 2003, Eppich et al

2006, Eich et al 2007b, Overly et al 2007) In our view, and according to our experience, they should also be suitable for the evaluation of guidelines in acute medicine when placed in sufficiently realistic clinical environments

Our research group at the medical simulation centres of Göttingen and Dresden aims to establish a model for simulator-based evaluation of clinical guidelines

in order to decrease the evidence gap prior to their implementation As a first pilot study of practical application, we compare a newly recommended controlled rapid-sequence-induction technique (RSI controlled) for non-fasted infants with the classic technique (RSI classic) using a high fidelity infant simulator (Eich et

al 2007a, Schmidt et al 2007) Using an observational checklist we record critical events and unsafe actions We simultaneously measure psycho-vegetative stress, based on ergospirometry (cardiorespiratory markers), saliva analyses (cortisol and alpha-amylase) and self-assessment (stress and safety perception)

The infant Simulator and its environment

For our current pilot study on RSI guidelines for infants, we work with a SimBaby™ infant simulator (Laerdal Medical), using software version 1.4 and its customary touch screen vital signs monitor The baby mannequin is placed

on an operating table in an authentic theatre environment within our simulation centre (see Figure 22.1)

SimBaby™ is a high fidelity integrated infant simulator which features all essential clinical and monitored vital signs It is controlled via computer interface

It has no physiologic model implemented but scenarios and trends can be pre-programmed For our RSI study, we programmed a standardized scenario on

anaesthesia induction in a four-week-old baby boy with pyloric stenosis Initial respiratory rate, oxygen saturation, heart rate, blood pressure, torso movement and vital signs monitor set-up were defined Additionally we programmed three trends for respiratory rate, heart rate and oxygen saturation: ‘RSI classic’, ‘RSI controlled’ and ‘recovery from hypoxemia’ (oxygen desaturation) All trends are started manually on induction of anaesthesia or on appropriate ventilation respectively The trends for oxygen saturation after breathing stops (apnoea) are based on oxygenation data derived from the Nottingham physiology simulator, as calculated for a one-month-old infant with no effective pre-oxygenation and open airway (Hardman and Wills 2006)

The Scenario

The techniques for RSI are standardized (see Figure 22.2)

The same anaesthesia nurse assists all procedures in an identical manner strictly following the protocol The induction drugs are drawn up in the predetermined dose and are administered by the anesthesia nurse when prompted by the candidate The nurse also provides exact timing to allow the first intubation attempt In combination with programmed scenario and trends, this procedure ensures that all candidates are

Figure 22.1 Study setting in theatres: infant simulator and anaesthesia work

station, anaesthesia nurse (left) and candidate (right) with the mobile ergospirometry unit applied