The Otolaryngology boot camp: a scoping review evaluating commonalities and appraisal for curriculum design and delivery

Abstract

Objective

Surgical boot camps are becoming increasingly popular in Otolaryngology–Head and Neck Surgery (OHNS) residency programs. Despite pioneering virtual reality and simulation-based surgical education, these boot camps have lacked critical appraisal. The objective of this article was to examine the adoption and utility of surgical boot camps in OHNS residency training programs around the world.

Data Sources

Ovid Medline and PubMed databases were systematically searched in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for scoping reviews. Additionally, a grey literature search was performed.

Review Methods

Inclusion criteria were peer-reviewed publications and grey literature sources that reported on OHNS boot camps for the novice learner. The search was restricted to human studies published in English. Studies were excluded if they were not examining junior trainees.

Results

A total of 551 articles were identified. Following removal of duplicates, screening, and full text review, 16 articles were included for analysis. Seven major boot camps were identified across various academic sites in the world. Most boot camps were one-day intensive camps incorporating a mixture of didactic, skill specific, and simulation sessions using an array of task trainers and high-fidelity simulators focusing on OHNS emergencies. Studies measuring trainee outcomes demonstrated improvement in trainee confidence, immediate knowledge, and skill acquisition.

Conclusion

Surgical boot camps appear to be an effective tool for short term knowledge and skill acquisition. Further studies should examine retention of skill and maintenance of confidence over longer intervals, as little is known about these lasting effects.

Introduction

Upon completing medical school, junior trainees enter post-graduate training programs with dramatically increased responsibilities. To address the concern regarding trainee skill inadequacy, surgical boot camps were developed to help develop skillsets from interpreting diagnostic imaging to performing surgical procedures [1].

The educational design of most surgical boot camps is a combination of didactic learning and small group simulation sessions. Both governing medical educational bodies of Canada (Royal College of Physicians and Surgeons of Canada) and the United States (Accreditation Council for Graduate Medical Education) have embraced competency-based educational frameworks for post graduate medical education (PGME) [2]. These frameworks are an outcomes-based approach to curriculum design where trainee advancement is dependent on mastering entrustable professional activities (EPA’s) [1]. With this shift, simulation training is integral in allowing trainees to practice clinical and procedural skills in areas specifically identified as key competencies or milestones before encountering real patient scenarios [1].

Literature examining the role of surgical boot camps has been extensively covered over the past decade. The majority of studies have examined the following outcomes: knowledge and technical skills acquisition, team communication skill development, and individual confidence improvement [3,4,5,6]. Moreover, surgical boot camps allow for social and cultural welcoming [7, 8]. Despite widespread adoption by various surgical specialties, including cardiac, general, neuro, orthopedic, trauma, and vascular surgery, few surgical boot camps have been reported on in Otolaryngology–Head and Neck Surgery (OHNS) [9,10,11,12,13,14,15,16]. Furthermore, OHNS boot camps lack critical appraisal despite being one of the leaders in virtual reality and simulation-based surgical education [17, 18]. The goal of this scoping review was to examine the utility of PGME surgical boot camps in OHNS around the world. To achieve this goal, this manuscript will address four fundamental objectives. (1) Thoroughly summarize existing OHNS boot camps around the world. (2) Determine overlap in curriculum design and delivery, resources, and simulation. (3) Examine pros and cons of existing boot camp formats. (4) Suggest an optimal boot camp design for junior residents in OHNS.

Methods

A scoping review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis scoping review (PRISMA-ScR) guidelines was performed in February 2021 [19]. The research databases included were Ovid Medline (September 1946 – February 2021) and PubMed (January 1946 –February 2021). The search terms included [(otolaryngology/otorhinolaryngology/ear nose throat/ENT/ORL/head and neck surgery) AND (“boot camp/bootcamp/training course)]. Inclusion criteria were peer-reviewed publications comparing pre- and post-course quantitative and qualitative data in skill performance or knowledge acquisition. The search was restricted to human studies published in English. In addition to the peer-reviewed search, an online grey literature search was utilized, specifically looking at conference proceedings and published information from medical educational and department websites. Excluded studies were non-English publications and studies not examining OHNS interns or junior residents (PGY-1 and PGY-2). Due to boot camps typically being introductory camps, the search was limited to junior residents. All other articles including opinion pieces and editorials were included for qualitative analysis.

Four reviewers (A.B-S., C.J.L., M.Y.H., & J.T.L.,) independently screened all abstracts to identify studies that fulfilled the predetermined eligibility criteria. Any disagreement between the reviewers was resolved by consensus. Qualitative data from each included study was extracted using standardized data forms including the study’s title, author(s), year of publication, education themes, and outcomes assessed.

Results

A total of 21 articles were identified by Ovid Medline, 527 articles by PubMed, and 3 articles from a grey literature search. Following the removal of the duplicate records, 530 abstracts were screened (Fig. 1) [20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94]. Of the 79 articles that underwent full-text review, 63 articles were excluded, and the remaining 16 articles underwent complete qualitative analysis with the data being summarized in Tables 1, 2, 3, 4 and 5.

Fig. 1

PRISMA flow diagram

Table 1 OHNS boot camps publications

Table 2 Learning objectives and common task trainers used in OHNS boot camps

Table 3 Boot camp simulators stratified by OHNS subspecialties

Table 4 Common didactic sessions and simulation scenarios in OHNS boot camps

Table 5 Common resources utilized in OHNS boot camps

Boot camps were analyzed for their course objectives, outcomes assessed, and overall study conclusions (Table 1). The earliest boot camp identified was in 2011, where Georgetown University (Washington, DC, United States of America) hosted the inaugural training course for junior trainees. This program established the standard to which subsequent boot camps developed their curricula [87]. Thirteen of the sixteen studies described one-day courses, while the remaining three were longitudinal in design, taking place over one- to six-months. All camps incorporated technical skills stations, simulation sessions, and didactic teaching surrounding common OHNS emergencies and consultation requests. Overall, these sixteen studies could more easily be organized into seven international boot camps with their associated academic centres (Tables 2, 4 and 5). Boot camps were based at the University of Georgetown (USA), New York University (USA), Albert Einstein College of Medicine (USA), University of California, Davis (USA), Western University (Canada), University of Cambridge (United Kingdom), and Hospital Un Canto a la Vida (Ecuador).

Most boot camps specifically stated their objectives. Common themes included recognizing and triaging common OHNS emergencies, performing critical basic procedural skills, communicating within a team, and knowing when to call for help. Several task trainers and simulators used for the development of specific procedural skills are listed in Table 2 and categorized by subspecialty in Table 3. The most common simulation scenarios included management of post-surgical and oropharyngeal bleeding (57%), acute airway obstruction from angioedema (43%), and facial/neck trauma (29%). The most common task trainers were surgical airway (71%), epistaxis (57%), peritonsillar abscess drainage (43%), and bag mask ventilation with tracheal intubation (29%). Taking this together, skills stations could be categorized into either 1) basic airway control or 2) special skills. Basic airway control stations include bag mask ventilation, intubation, and surgical airway simulation. Special skill stations include bronchoscopy, peritonsillar abscess drainage, epistaxis and post-tonsillectomy hemorrhage control. Using this terminology allows boot camps to develop goal-oriented simulation stations with thoughtful and explicitly stated objectives.

With respect to each boot camp’s educational frameworks, all courses incorporated some elements of didactic and simulation sessions (Table 4). Didactic sessions involved common OHNS on-call scenarios, emergency situations, operative skills, and perioperative care of the post-surgical patient. Simulation sessions were predominantly focused on acute and subacute OHNS presentations including airway obstruction, epistaxis, and trauma. OHNS simulation resources can be subdivided into physical task trainers to virtual reality platforms [17]. Physical task trainers including mannequin, animal, and cadaveric simulators are often employed (Table 5).

Our synthesis of the data demonstrated that participation in introductory boot camps appears to improve trainee confidence [16, 84, 87, 93], immediate knowledge acquisition [82, 85, 92, 93], and immediate improvement in procedural skills [83, 91] (Table 1). Studies utilizing prospective cohorts and randomized controlled trials (RCTs) revealed an improvement in immediate didactic knowledge (as demonstrated by multiple choice examination), technical skills (based on blinded faculty assessment), and self-perceived confidence which was maintained up to 6 months [82, 87, 91, 93]. In a head-to-head RCT comparing simulation versus traditional didactic learning methods, junior trainees randomized to the simulation arm performed significantly better in both epistaxis and epiglottitis scenarios scored individually by a blinded expert surgeon. Additionally, participants randomized to the simulation group had an improved perception of education and were more likely to make positive recommendations to their colleagues [91].

This study is the first scoping review in OHNS boot camps for junior resident learners. Through our analysis, we have gained valuable insight into the variability of practices around the world. In Table 6 and Table 7, we have summarized our interpreted pros and cons of various boot camps features and developed suggestions for successfully implementing an OHNS surgical boot camp for junior residents.

Table 6 Pros and Cons of various boot camp features

Table 7 Keys to success for OHNS boot camps.

One-to-seven-day camps for junior learners provide an optimal balance of relative ease in camp set up and execution with less time away from clinical activities for learners. Multidisciplinary staff including faculty from anesthesia, emergency medicine, thoracic surgery along with OHNS may provide added expertise and allow for more focus on interdisciplinary teamwork which is integral for trainee development. Didactic-based curriculum leads to improvements in knowledge retention and comprehension post course [82] while simulation improves confidence, competence, skill performance, and adds value to the learners’ overall experience with specific emphasis on teamwork / collaboration [1, 82, 85, 87, 91, 94]. Therefore, a curriculum with both didactic and simulation-based learning is advised. Learner feedback should be facilitated in a safe learning environment with emphasis on resident experience with combination of structured written and oral debriefing sessions after simulation (Tables 6 and 7).

Discussion

Intensive crash courses for residents and fellows have existed in OHNS for numerous years employing simulation to enhance specific aspects of training such as functional endoscopic sinus surgery, removal of foreign bodies, or management of facial trauma [26, 32, 77, 78]. Contrastingly, the concept of an introductory “boot camp” style training course for incoming OHNS trainees emphasizing fundamental skills is a recent occurrence.

As the first published modern-day boot camp for junior OHNS trainees, the Georgetown University boot camp began as a simple, simulation-based one-day emergency course. This has become popularized across the world since its inception in 2011 [87]. Many institutions have adopted similar boot camp style courses for junior trainees with mirroring objectives and content throughout the United States, Canada, and the United Kingdom. Several themes of the modern-day boot camp include the use of simulation, interdisciplinary faculty and trainees (anaesthesia, emergency medicine, family medicine, and pediatrics), and the use of validated educational frameworks for curriculum design (Kolb learning style theory and needs assessment models).

Simulation is an educational approach that enables learners to encounter components of the clinical interactions while enabling educators to provide education and simultaneous assessment in a standardized environment [17, 95]. Widely adopted across various industries, simulation as a training adjunct has become a staple in aerospace and military training, whereas its adoption in medical education has been comparatively slow [96]. In 2012, the Accreditation Council for Graduate Medical Education (ACGME) recognized simulation as a means of evaluating resident performance for various “educational milestones,” in its shift towards competency-based medical education [97, 98]. In OHNS, trainee knowledge and procedural skill were evaluated via cadaveric dissection, temporal bone drilling, and surgical simulator labs [99]. A recent national survey of American OHNS residency programs demonstrated that nearly two-thirds of programs incorporated simulation modalities into curricula [100]. When assessing the Canadian landscape in OHNS programs, 30.8% actively use some form of VR training simulator that 90.9% of program directors felt would be a fair and effective means for evaluation [101]. Given the importance of simulation training in OHNS, many boot camps utilize this method to help junior trainees develop critical skills in a controlled environment.

In this scoping review, all seven boot camps used simulation as the curriculum core through simulation scenarios and specific task trainers. The most common simulation scenarios included management of post-surgical and oropharyngeal bleeding (57%), acute airway obstruction from angioedema (43%), and facial/neck trauma (29%). The most common task trainers were surgical airway (71%), epistaxis (57%), peritonsillar abscess drainage (43%), and bag mask ventilation with tracheal intubation (29%). High fidelity cadaveric and mannequin-based task trainers for task specific procedures appear to be the current trend. All studies that used high fidelity simulation scenarios used the Laerdal (Wappinger Falls, NY) SimMan® adult simulator. SimMan® offers a highly realistic training model with real time neurological and physiological function.

Despite some of the diversity in task trainers and simulations used across the world, the principal theme in all boot camp curricula appeared to be management of emergency situations and on-call scenarios. The goal was to have junior trainees leave the camp equipped with the skillset to identify and triage acute emergencies, perform basic minor airway procedures, and communicate and activate emergency protocols. We noted that trainee participation in introductory boot camps appears to improve their confidence, immediate knowledge acquisition, and immediate improvement in procedural skills in comparison to traditional didactic methods of learning [82, 87, 91, 93]. Simulation learning also improved performance significantly in epistaxis and epiglottis scenarios, improved perception of education and increased the likelihood of making positive recommendations to colleagues when compared to traditional didactic learning methods [91]. The large heterogeneity of the studies included in this review precludes meta-analysis. However, the role of this scoping review was to examine OHNS boot camps more descriptively around the world. Here we have identified a trend in the literature suggesting positive outcomes for trainees that participate in introductory boot camps for their overall clinical and psychosocial development as an early trainee.

Despite strongly positive outcomes from boot camps and simulation training, criticisms of the lack of evidence to suggest long-term retention exist [31, 67]. Three studies demonstrated that perceived confidence in procedural tasks and knowledge lasted up to 2- 6 months [87, 93, 102]. However, neither long-term knowledge retention nor procedural competency has been assessed among OHNS trainees. Also, according to a survey of OHNS residency program directors in the United States and Puerto Rico, there are several barriers that exist which prevent participation in boot camps and simulation training [67]. Some of these include cost, lack of local access, lack of interest, and scheduling difficulties [67]. This suggests making boot camp programs more widely available, having partially subsidized costs, and more data on their short- and long-term benefits could address the hesitancy that some program directors have.

Although boot camps are typically delivered at the beginning of OHNS programs because they are introductory, consensus on when they should be offered is lacking. When surveying American OHNS program directors, a slight majority felt boot camps should be offered within the first few months of residency [67]. Interestingly, simulation training programs have been shown to be effective in all postgraduate years, with knowledge and skills acquisition demonstrated across all training levels [31]. Several other studies have evaluated the effectiveness of OHNS boot camps for medical students and suggest that boot camps may aid with the transition to residency as they all reported improved knowledge, confidence, and clinical performance after completion of the course [27, 33, 38, 62]. Taking these pieces of evidence together, it seems that the surgical boot camp style of education delivery at any level is beneficial in the short-term of less than six months. The lasting effects, however, remain uncertain and future investigations should examine the long-term retention of knowledge, confidence, and technical skill.

Conclusion

Boot camp style training programs for junior OHNS are becoming widely adopted across the world. Fuelled by the utilization of simulation technology to deliver time-effective education for common OHNS emergencies, these programs embrace the educational shift towards competency-based accreditation standards for residency programs. A number of studies have justified this form of education to improve trainee’s performance, confidence, and skill in the short term. However, current literature has failed to examine a number of important long-term outcomes. Future studies that examine the effect of OHNS boot camps on long term outcomes will play a critical role justifying widespread adoption of boot camps for resident education.