In-situ simulation-based team training reduces incidence of negative events during bronchoscopy. A prospective educational intervention cohort study

Abstract

Introduction

While bronchoscopy complications are rare, they can be life-threatening if not quickly managed. This study evaluates the effect of a case-based bronchoscopy simulation training using real-world data on complication incidence and nature.

Methods

Based on semi structured interviews with respiratory staff in a bronchoscopy unit a team simulation training case was constructed. It was assessed using the Kirkpatrick framework to measure changes in procedural behavior by the rate of adverse events (level three) as the main outcome. Participants’ reactions, changes in stress levels, and patient perspectives (levels one, two, and four) were evaluated via questionnaires.

Results

Following the educational intervention, the incidence of any negative events during bronchoscopies was reduced from 62% (38/61) to 41% (26/63), p = 0.019. The most frequent event was oxygen desaturation below 90%, which occurred in 34% of the bronchoscopies before the intervention vs. 11% afterwards, p = 0.002. The participants found the simulation-based training relevant but did not change the perceived level of stress. The patient reported to be less awake (2, IQR 1–5, vs. 5, IQR 3–8), p = 0.02 after the intervention.

Conclusion

Incorporation of in-situ simulation-based team-training for crisis management during bronchoscopy alter procedural behavior and significantly reduce the occurrence of adverse events; therefore, it should be integrated into future bronchoscopy training curricula.

Introduction

Bronchoscopy constitutes an invasive procedure prevalently employed in the field of respiratory medicine, serving as a fundamental procedure in the diagnosis of various respiratory diseases, including lung cancer, infections, and interstitial lung diseases. This procedure can be conducted as either flexible bronchoscopy or endobronchial ultrasound (EBUS), facilitating the visualization of the bronchial tree and enabling biopsy of potentially suspect lesions [1, 2].

Bronchoscopies are generally considered safe; however, adverse events can occasionally occur and, if not promptly and adequately addressed, may progress to life-threatening conditions and potentially result in fatality [3, 4]. Patients commonly undergo mild sedation during a bronchoscopy. Given that many patients have preexisting compromised pulmonary function, sedation can present a risk of respiratory depression and hypoxemia, particularly in frail individuals [5,6,7]. Moreover, the bronchoscopy procedure itself poses risks of hypoventilation, pneumothorax, endobronchial bleeding, and bronchospasm [8,9,10]. Consequently, it is imperative that the bronchoscopy team is proficient in detecting, interpreting, and responding to adverse events associated with bronchoscopy, as these events, despite their rarity, are potentially alterable and can lead to significant consequences if not managed promptly and effectively.

Simulation-based training is advocated to enhance team performance in procedures where high-stress situations are infrequent but have the potential to be catastrophic, such as anesthesiologic airway management techniques [11, 12]. While simulation training in bronchoscopy has been primarily focused on improving technical skills to operate the bronchoscope, navigate the bronchial anatomy, and recognize critical landmarks for biopsy planning, in situ simulation-based training may also offer an opportunity to cultivate intra- and interpersonal skills crucial for managing adverse events associated with the procedure. Such a training format facilitates participants’ immersion in a quasi-realistic experience of the procedure, and scenarios can be constructed to incorporate key clinical components, including the identification of complications, enhancement of interpersonal skills such as team communication, and development of intrapersonal competences like clinical decision-making processes, selecting an appropriate response and strategy for complication management [12, 13].

Although possessing potential advantages, the application of in-situ simulation-based training to moderate negative events in bronchoscopy remains unexplored. Incorporating such training may enable the bronchoscopy team to maintain vigilance and preparedness to manage adverse events associated with the procedure, enhance clinical decision-making, and ultimately elevate patient satisfaction and safety by diminishing the incidence and impact of negative occurrences. Consequently, the aim of this study was to develop and implement a simulation-based in-situ training module designed to address adverse events in bronchoscopy, and to assess the effectiveness of this intervention on staff responsiveness, alterations in procedural conduct, the incidence of negative events, and the perspectives of patients undergoing bronchoscopy.

Methods

Design

The study was designed as a prospectively longitudinal interventional education study with real-world data collection. The SQUIRE-EDU and INSPIRE guidelines for reporting educational health projects and simulation-based interventions were followed [14, 15].

Context

The study was carried out in the Department of Respiratory Diseases and Allergy at Aarhus University Hospital in Denmark. The department performs approximately 2.200 pulmonary endoscopies yearly, including flexible bronchoscopy with endobronchial biopsies, bronchoalveolar lavage, nodular biopsies guided by electromagnetic navigation, fluoroscopy and radial EBUS (Endobronchial Ultrasound), parenchymal cryo-biopsies, linear EBUS with needle aspiration and lymph node cryo-biopsies, EUS (esophageal ultrasound), EUS-b (EBUS biopsies through the oesophagus), endobronchial lung volume reduction, and whole lung lavage. Endoscopies are performed under sedation with midazolam and fentanyl as standard. General anesthesia is used in some procedures. Advanced endoscopies are performed by consultants with a special interest in interventional pulmonology. Residents perform flexible bronchoscopy with BAL and endobronchial biopsies with or without supervision according to their level of expertise after completing a standardized national bronchoscopy educational program that adheres to international guidelines [6, 7, 16, 17]. Highly trained nurses employed in the unit assist bronchoscopies.

The resources used for a bronchoscopy include a physician and two nurses, of whom one handles sedation, observes, and interacts with the patient during the procedure, and one who assists the physician in handling biopsy tools. A cytological technician performs a rapid on-site cytological evaluation (ROSE) of needle aspirates if these are taken.

Intervention

The simulation case was developed by the authors, who were all certified instructors in simulation-based training [18]. The curriculum development was performed using Kern’s six-step model to educational development [19]. The six steps are 1) Problem Identification and General Needs Assessment, 2) Targeted Needs Assessment 3) Goals and Objectives, 4) Educational Strategies, 5) Implementation, 6) Evaluation and Feedback. In regards to step 1 a national needs assessment in 2016 in pulmonary medicine placed flexible bronchoscopy highest of technical procedures that should be training using simulation-based training [20]. The content of the case (step 2 and 3) was chosen based on semi-structured interviews with nurses employed in the endoscopy unit and respiratory medicine consultants. They were asked about the most frequent negative event during bronchoscopy, the most serious negative event, how communication on the team affects handling of negative events and how management of these events could be improved, and how team communication could be improved. Step 4 and 5 was in-situ based simulation as it was stressed that simulation was a suitable tool for accommodating the learning goals and that it should be conducted in the real clinical environment in order to enhance feasibility and transferability of the training. Step 6 is done based on the analysis of the present paper.

Case

The case included three consecutive elements that each needed to be completed to proceed in the case. The instructors judged if the team had reacted satisfactory and then proceed to the next element. In the first element, the patient had a severe cough and motor agitation that required additional doses of sedation for the bronchoscopy to continue. In the second element, the additional sedation caused the patient to have respiratory depression, which required adjustment and administration of the oxygen supplement, and improvement of oxygen saturation. If the team handled this well, the case continued with a biopsy procedure that caused severe endobronchial bleeding that the team needed to manage accordingly. If adequate action was taken, the case was completed; if not, the patient developed fulminant respiratory failure, and the resuscitation team was called. The case, including information for instructors, is available in Supplementary file 1.

Simulation-based training

The participants, comprising either nurses employed in the Department of Endoscopy or respiratory physicians, fulfilled their professional roles within the scenario, resulting in a team composition of one physician and two nurses, which is the standard practice at our center. Each participant engaged in the training session only once. The training had a total duration of approximately forty-five minutes. For further details, see Supplementary Table 1.

The instructor first read aloud a five-minute briefing on how the simulation was to be performed, the patient’s history, the results of a CT scan, and the indication for bronchoscopy. Then the scenario started.

Simulation training was conducted in the bronchoscopy suite and all remedies used in a real-life bronchoscopy, including personal protection equipment, a bronchoscope, biopsy tools, medications, syringes, oxygen delivery devices, observation equipment, etc., were available in their normal places to maximize simulation realism.

A mannequin phantom (Laerdal, Resusci Anne QCRP, Full body) was employed in the case. Vital signs (such as pulse, blood pressure, oxygen saturation, and respiratory rate) were displayed on a monitor. During the scenario, modifications in the patient’s condition were verbally communicated by the instructor (e.g., “the patient is now coughing,” “the patient is now agitated,” etc.), as the mannequin was incapable of exhibiting these behaviors autonomously. A video recording of a real and uneventful bronchoscopy was presented on a bronchoscopy monitor. The film was unable to exhibit events such as endobronchial bleeding; if such occurred during the simulation, verbal instruction was provided to the team.

Interventions by the team was executed in real-time to enhance the authenticity of the scenario. Should the team decide to administer medication, they were required to get the actual medication from its designated storage. However, rather than opening the real medication containers, participants utilized small saline bottles, drawing the solution into syringes, which were then injected into an intravenous access device that was attached adjacent to (yet not inserted into) the phantom.

Finally, a fifteen minute debriefing was done to discuss the course of the case, the clinical problems that occurred, the views of the teams on their performance and the communication to facilitate the teams’ learning process. The debriefing was based on the GAS model that includes a Gathering information phase, an Analysis of the training and a Summarization and discussion on transferability into clinical practice [21].

Study of intervention

The Kirkpatrick four elements methodology was used as a framework to evaluate the effect of training [22]. This framework was modified to entail elements relevant to the study aims: Kirkpatrick level one, the reaction: Participants evaluated their reaction to training in a questionnaire shortly after completion of it. See supplementary Table 2. Kirkpatrick level two, the learning, change in skills, attitude, and confidence: The participants evaluated the perceived level of stress in different critical bronchoscopy situations ranging from small nose bleeding to respiratory and cardiac arrest. Each situation was evaluated before and after the simulation training on a ten-point scale. See supplementary Table 3. Kirkpatrick level three, behavior in the clinical setting: All events that occurred during real-life bronchoscopy procedures were recorded before and after the intervention. It was recorded that the procedure was completed as planned, that unexpected events occurred, and that unplanned actions were taken during the procedure. See Supplementary Table 4. Kirkpatrick level four, results: The patients who underwent bronchoscopy rated their experience in twelve questions; see Supplementary Table 5, grouped into a preprocedural domain, an intraprocedural domain, a postprocedural domain, and a general domain before leaving the postprocedural observation room. Data on patient demographics, pulmonary conditions, and bronchoscopy interventions were documented and compared pre and post the training intervention to evaluate confounding variables that might account for observable differences.

Measures

The primary outcome measure was the difference in the rate of any unexpected or unplanned adverse events occurring during bronchoscopy (Kirkpatrick level 3 and 4) before and after the training intervention. Bronchoscopy nurses reported all predefined events in a questionnaire after each bronchoscopy. See the questionnaire in Supplementary Table 4.

Secondary outcomes included

Reaction to the training intervention (Kirkpatrick level 1) evaluated in a questionnaire filled out by participants after the training, Supplementary Table 2. Change in attitude and confidence in critical bronchoscopy situations (Kirkpatrick level 2) before vs. after the training intervention reported in a questionnaire, Supplementary Table 3. Differences between job functions in these changes. Changes in patient-reported outcomes (Kirkpatrick level 4) before vs. after the training intervention reported by the patients who had undergone bronchoscopy before they left the hospital, Supplementary Table 5.

Ethical considerations

The bronchoscopy staff participated voluntarily in the training intervention, and this followed the normal standard to generate a safe learning space, a high level of comfort and security, and a feeling of confidence as other local training interventions. The patients underwent bronchoscopy independently of the training intervention and filled the questionnaire voluntarily.

Statistical considerations

Results are presented in numbers and percentages, means and standard deviations, or as medians with interquartile ranges as appropriate to parametric distribution which was assessed by quartile-quartile plots and histograms. The difference between continued measures was evaluated with the unpaired student t-test or the Wilcoxon-Mann–Whitney unpaired rank test. The difference in incidence rates was analyzed with the Chi-square test or the Fischer’s exact test. Missing data were not imputed and were considered missing at completely random. The significance level was established at 5% for individual measurements; the primary outcome. However, for secondary outcomes that involve multiple testing, the Bonferroni correction was applied, resulting in a significance level of 0.3% for this analysis.

The sample size was estimated for the primary outcome, the incidence of any negative events during bronchoscopy, and a minimum relevant difference was established a priori at 10%. With an estimated incidence of 30%, a Wald test found that a total of 126 bronchoscopies needed to be captured.

Results

Thirteen bronchoscopy personnel participated and completed the simulation training intervention of which 69% (9/13) were nurses and 31% (4/13) were physicians. Most of the participants, 92% (12/13), were women and only 8% (1/13) were men.

A total of 124 patients underwent bronchoscopy and were included in the study. Almost all, 98% (121/124), of the bronchoscopies were completed as planned. In 49% (61/124) of the patients, the bronchoscopy was performed before the training education, and in 51% (63/124) it was done afterward. The median age was 71 years (IQR 56 – 85) and 54% were men. There was no difference in WHO performance status, lung functions parameters, TNM-stadium, nature or number of procedures performed, before versus after the training intervention, as shown in Supplementary Tabel 6.

Primary outcome

At Kirkpatrick level three, any negative events occurred in 52% (64/124) of all bronchoscopies. In 30% (37/124) of the bronchoscopies, only one negative event occurred but in 16% (20/124) two negative events occurred, 4% (5/124) had three events and 0.8% (1/124) had four or more simultaneous events.

Following intervention training, the incidence of any negative events was reduced from 62% (38/61) to 41% (26/63), p = 0.019, corresponding to a relative decrease of 32%.

As seen in Table 1 the change in the incidence of negative events was highest for the event of oxygen saturation below 90% and there were no immediate life-threatening events such as bronchospasm, pneumothorax, respiratory arrest, or cardiac arrest.

Table 1 Incidence and nature of unexpected and unplanned outcomes before and after the training intervention

Secondary outcomes

The reaction to simulation-based training (Kirkpatrick level 1) was positive as shown in Fig. 1. On a scale from one (low) to five (superior) the participants rated in mean that it was fun to participate in the simulation: 4.3, (SD = 0.7), that they felt safe to participate in the simulation: 4.6, (SD = 0.7), that the simulation was usable for clinical practice: 4.2, (SD = 0.8), that the simulation was relevant to clinical practice: 4.5, (SD 0.6), that the case was realistic: 4.1, (SD = 0.8), that their personal competences improved as a result of the simulation: 3.8, (SD = 0.7), and that the entire bronchoscopy team improved their competences: 4.4, (SD = 0.7) as a result of the simulation training.

Fig. 1

Percent distribution of questions on the perceived effect of the simulation. The participants rated questions on a scale from one- lowest- to five -highest. One: no data, two: no data, three: blue, four: orange, five: green (color figure online)

The perceived stress level, attitude, and confidence in critical bronchoscopy situations (Kirkpatrick level 2) did not change by simulation training in any of the scenarios shown in Fig. 2. Nurses had a higher perceived stress level than physicians if the patient was motoric agitated (difference 1.9 (95% CI 0.5–3.4), p = 0.01), if the patient had pneumothorax (difference 2 (95% CI 0.4–3.6), p = 0.02), and if the patient needed oral suction (difference 2 (95%CI 0.2–3.8),p = 0.03) during bronchoscopy. There were no scenarios in which the perceived stress level was higher for physicians than for nurses. The level of perceived stress on nurses was higher when assessed by themselves than when assessed by physicians in all scenarios and the difference reached the statistical significance level in the scenarios in which patients had severe cough (1.9 (95% CI 0.3–3.4), p = 0.02), if the patient was motoric agitated (1.5 (95% CI 0.1–2.9),p = 0.04), and if the patient had pneumothorax (2.6 (95% CI 0.8–4.3), p = 0.01) during bronchoscopy. Similarly, the physicians rated that the perceived level of stress was higher for the nurses than for the physicians themselves in all scenarios and the difference was statistically significant in the scenarios where the patient was motoric agitated (2 (95% CI 0.4 – 3.5), p = 0.02), if the patient needed oral suction (1.8 (95% CI 0.3—3.3), p = 0.03), and if oxygen administration needed to be adjusted during the bronchoscopy (2.1 (95% CI 0.3 – 3.9), p = 0.03).

Fig. 2

Perceived level of stress on the bronchoscopy team on a scale ranging from zero (lowest) to ten (highest) on different scenarios that may occur during bronchoscopy before and after the simulation training

At Kirkpatrick level four, patients who underwent bronchoscopy after the simulation training was completed reported lower level of consciousness during bronchoscopy (median 2, IQR 1 – 5), than patients who had bronchoscopy before training (median 5, IQR 3 – 8), p = 0.02. There was no change in the level of general unpleasantness, cough, dyspnea, chest pain, throat pain, overall satisfaction, feeling of comfort during and after the bronchoscopy, or feeling it was safe to leave the hospital after recovery, reported by the patients, as seen in Fig. 3.

Fig. 3

Patient-reported outcomes following the bronchoscopy on a scale from one (lowest) to ten (highest). Patients reported to be significantly less awake during the bronchoscopy after the simulation training was completed. There was no change in the other outcomes

Discussion

In this study, an in-situ simulation training intervention was developed to address complications associated with bronchoscopy. The implementation of the training was assessed based on its impact on the incidence of adverse events, the perspectives of staff regarding perceived stress, and the experiences of patients undergoing the bronchoscopy procedure.

The primary outcome of the study indicates a significant reduction in the frequency of adverse events following the training intervention. This decrease was largely attributed to a substantial reduction in the incidence of patients experiencing episodes of mild oxygen desaturation (below 90%, though not less than 80%). The simulation case was designed to incorporate training specifically focused on the management of mild desaturation events. Notably, patients reported a lower level of consciousness during bronchoscopy subsequent to the training intervention. This suggests that the observed decrease in mild desaturation events cannot be attributed to lighter sedation levels. It is plausible that the training program heightened the awareness regarding oxygen saturation, which facilitated an earlier modification of the oxygen dosage by the bronchoscopy nurse.

In general, a greater number of adverse events were documented during the study period than has been previously reported [3, 4, 8]. This observation is likely attributable to the prospective design of our study, which ensured that all predefined events were systematically recorded. The overwhelming majority of the adverse events were mild, and none of the most severe and life-threatening events were observed during the study period. Only a small number of patients experienced severe oxygen desaturation (below 70%), and the incidence was not influenced by simulation training. Additional factors, beyond the enhanced team awareness and focus on event identification fostered by the simulation training, are likely accountable for the few severe events noted in this study. Patient-related factors such as the degree of morbidity and lung function status are unaffected by a simulation training program and therefore are unlikely to be modifiable through such an intervention.

At the lower levels of the Kirkpatrick framework, participants of the training program expressed that the simulation training was of considerable utility and relevance, asserting its significance to practical application. Nonetheless, they reported unchanged levels of perceived stress concerning a range of pre-selected negative events associated with bronchoscopy. Consequently, although the participants exhibited a favorable reaction to the simulation training, it appears not to have resulted in an alteration of their self-assessed competence to manage these events. Particularly, the severe and immediately life-threatening events, such as pneumothorax, substantial bronchial hemorrhage, and respiratory or circulatory collapse, remained profoundly stressful scenarios for the participants.

The limitations of the study include that the sample size was insufficient to capture of any very severe adverse bronchoscopy events during the study period. These events are, fortunately, infrequent, and capturing them would have required a substantially extended study duration, which was beyond the scope of our study capabilities. Another limitation is that the evaluation tool employed to assess changes in perceived stress, were developed specifically for this study and had not undergone any validation process. As no tool specific to bronchoscopy currently exists, our instrument was adapted from other endoscopic or surgical evaluation tools [23,24,25,26]. Data regarding the patients’ perspective was lost as several patients departed the recovery room prior to completing the questionnaire. A third limitation pertains to the demographic composition of the training intervention participants, with a predominance of female participants. It is conceivable that male participants might exhibit a different response to receiving feedback. Additionally, the fourth limitation involves the management of endobronchial bleeding, which adhered to local standards as part of the case. It is acknowledged that alternative management approaches, such as balloon catheters or double lumen intubation, might be employed in other centers; however, these were not included in the present scenario. Fifthly, we tried to maximize the fidelity of the simulation, although it did not permit the inclusion of all aspects (motoric agitation, coughing, real-life change in bronchoscopy video, etc.). However, the participants rated the level of realism as high. Finally, considering the pre-and-post study design, it is not possible to completely rule out the influence of confounding variables on the results. However, no differences were observed in the variables of patient age, sex, or performance status, nor in the procedures performed prior to and following the training intervention. A key strength of our study is the application of the Kirkpatrick framework to evaluate the effect of the systematic training intervention across various organizational levels. Utilizing this evaluation framework allowed us to assess levels ranging from participants’ reactions, alterations in perception, changes in clinical practice, to modifications in patient perspective. Reporting the effects of a training intervention on all these levels is uncommon and constitutes a significant strength of our study [27].

In perspective, the educational dimensions of bronchoscopy have been subject to significant development in recent years. Formal educational programs, incorporating validated assessments, have transformed the training paradigm from the traditional ‘see one, do one, teach one’ philosophy to simulation-based training that ensures a fundamental level of technical competence is achieved prior to a bronchoscopist undertaking the procedure on an actual patient for the first time [24,25,26, 28]. Future training could potentially encompass crisis management, an understanding of potential complications, decision-making, and appropriate responses to adverse events during bronchoscopy and it could be speculated that training should encompass the entire bronchoscopy team and be administered in-situ to accurately represent the complexities encountered during actual crises in bronchoscopy, allowing for an assessment of team reaction and performance alongside the practical and facility considerations of the bronchoscopy suite. Although the findings of our study do not substantiate the formation of such recommendations, it is imperative that future research be conducted to assess the significance of non-technical elements in the management of crises related to bronchoscopy complications.

In conclusion, this study demonstrates that the implementation of an in-situ simulation team-training intervention, designed to enhance the focus on patient safety during bronchoscopy, enabled the bronchoscopy team to reduce frequency of minor complications, thereby contributing to improved patient safety.