Educational aims
The reader will come to appreciate that Simulation Based Education:
- •
Is an important tool in delivering high quality, evidence-based education to healthcare workers.
- •
Should be evaluated against real patient outcomes and have led to significant improvements in the care of patients in neonatology and paediatrics.
- •
Can be used the enhance clinical skill acquisition, acute care management, teamwork, leadership, communication and for quality improvement.
Abstract
There is increasing use of clinical Simulation Based Education (SBE) in healthcare due to an increased focus on patient safety, the call for a new training model not based solely on apprenticeship, a desire for standardised educational opportunities that are available on-demand, and a need to practice and hone skills in a controlled environment. SBE programs should be evaluated against Kirkpatrick level 3 or 4 criteria to ensure they improve patient or staff outcomes in the real world. SBE programs have been shown to improve outcomes in neonatology – reductions in hypoxic ischaemic encephalopathy, in brachial plexus injury, rates of school age cerebral palsy, reductions in 24hr mortality and improvements in first pass intubation rates. In paediatrics SBE programs have shown improvements in paediatric cardiac arrest survival, PICU survival, reduced PICU admissions, reduced PICU length of stay and reduced time to critical operations. SBE can improve the non-technical tasks of teamwork, leadership and communication (within the team and with patients and carers). Simulation is a useful tool in Quality and Safety and is used to identify latent safety issues that can be addressed by future programs. In high stakes assessment simulation can be a mode of assessment, however, care needs to be taken to ensure the tool is validated carefully.
What is Simulation Based Education?
The effectiveness of Simulation Based Education (SBE) depends on how well it is used. Simulation should be used as an adjunct to patient care experiences, and its use as an education tool should be planned and outcome driven . Simulation is one tool to use in educating individuals and teams in the care of sick children .
When reviewing the clinical effectiveness of SBE programs, the four level Kirkpatrick model is often considered. Although programs showing changes in learning and behaviour are important, the ultimate aim is to develop simulation education that shows an improved outcome in patients , Fig. 1 . This review will focus on Level 3 and 4 outcomes where possible.
Yousef and Soughier discuss the disadvantages of clinical vs. simulation based intubation training in neonates, which also apply to other clinical skills and environments, and are illustrated in Table 1 .
Demonstrating the need.
Training for High Acuity, Low Occurrence (HALO) events is particularly amenable to SBE. HALO events can be clinical presentations, including neonatal resuscitation , anaphylaxis or clinical procedures such as Emergency delivery, thoracotomy and pericardiocentesis .
Maintaining mastery is essential for specialists and SBE can help achieve this. A survey of 50 practising ENT surgeons in the UK who were responsible for providing a paediatric emergency rigid bronchoscopy service found that 38 % had not performed paediatric bronchoscopy in more than 5 years and yet when asked about personal confidence, 48 % were extremely high or high, 32 % somewhat, 12 % not so and 6 % not at all confident .
In some circumstances HALO events in one jurisdiction can be much more frequent in another. The need for SBE to ensure quality care is even more important. A 2023 South African study showed 71 % of paediatric junior doctors had participated in > 10 resuscitations in the last 2 years, but only 39 % had attended an accredited resuscitation course in the last 12 months and 42 % of respondents stated they had never received any formal training nor exposure to simulations in the management of seriously ill and injured children . Gal et al. studied 228 paediatric trauma resuscitations at an American Level 1 trauma centre. Only 22 % of primary surveys and 0 % secondary surveys were completed correctly .
Improvements in paediatric patient outcomes using SBE
One of the most evaluated and proven effective SBE programs demonstrating improved outcomes for children is the Practical Obstetric Multi-Professional Training (PROMPT) program. This program is a ‘course in a box’ approach to obstetric emergencies using a multidisciplinary team training. The program has been evaluated worldwide and improvements in care include a 50 % reduction in hypoxic ischaemic encephalopathy (HIE), 100 % reduction in permanent brachial plexus injury, a 45 % reduction in school age cerebral palsy, and 40 % quicker birth at Category 1 caesarean section . The course has also had extensive cost analysis and although the multidisciplinary full day courses are expensive from a personnel perspective, the saving from insurance payouts have been considerable and have led to ongoing funding of the program .
The ICU-RESUScitation Project examined the association between a CPR simulation program and simulated and actual performance during paediatric in-hospital cardiac arrest in 11 PICUs. Actual CPR performance was worse than simulation in achieving target rate, compression depth and chest compression fraction. Characteristics associated with improvements in achieving high quality CPR and/or improvements to survival to hospital discharge were more recent participation in simulation, a higher proportion of nurse participation and simulations conducted on weekdays .
The “Helping Babies Breathe” SBE program combining a one-day training course, with weekly 3 min facilitated practice and 40 min facilitated retraining monthly. An evaluation study in Tanzania showed a significant increase in the number of stimulated neonates and infants suctioned, with a reduction in the need for bag and mask from 7.3 % to 5.9 % (p = 0.005). The 24 hr mortality rate reduced from 11.1/1000 to 7.2/1000 (p = 0.04) .
Sawyer et al. evaluated the impacts of an interdisciplinary paediatric extracorporeal CPR (eCPR) simulation program. They demonstrated a reduction in activation time from 7 to 2 min. The 24 h survival increased from 45 % to 56 %, and the survival to discharge increased from 27 % to 38 % . Similar results were produced by Di Nardo et al. .
The provision of paediatric mock codes has been shown to significantly correlate with improved paediatric cardiac arrest outcomes. Clinicians responded to mock codes randomly called at increasing rates over a 48-month period, just as they would an actual event. Survival rates increased from 33 % to approximately 50 % (p < 0.001), correlating with the increased number of mock codes (r = 0.87). These results were maintained for 3 consecutive years .
Improvements in paediatric clinical procedures using SBE
An evaluation of a 5-year neonatal simulation program in Qatar showed that first attempt intubation success in real patients improved from 27 % to 76 % (p = 0.001) .
Evaluation of a four-hour advanced neonatal skills simulation-based workshop showed immediate improvements in Direct Observation of Procedural Skills (DOPS) scores for chest tube insertion, defibrillation, exchange transfusion, IO access, paracentesis and pericardiocentesis. All procedures had skill decay when measured again at 9–12 months, except for chest tube insertion, and all were improved with a further intervention . This study also demonstrates the discordance between self-perceived competence and preceptor rated DOPS. Apart from pericardiocentesis which showed no change, self-perceived competence increased between post intervention testing and retesting at 9–12 months. However, DOPS scores fell for all skills except for chest tube insertion.
Manikins can be used to train in paediatric laryngoscopy/bronchoscopy intervention and foreign body management. American data has identified that clinical exposure for ENT residents to paediatric foreign body airway obstruction occurs on average 1.3 times a year which isn’t enough to gain clinical mastery . A simulation course using high fidelity manikins and objective assessments such as an Objective Structured Assessment of Technical Skills (OSATS) tool have demonstrated skill acquisition to the level of ‘independent without errors’ . Virtual reality (VR) bronchoscopy training has also been shown to improve speed of bronchoscopy, dexterity and accuracy . Jiang et al. compared VR simulation and manikin simulation for novices and found VR simulation to be more efficient in achieving plateau of learning curves in bronchoscopy . A five minute VR “warm up” has also been shown to significantly improve performance in residents performing fibreoptic intubation in live patients .
The introduction of SBE doesn’t always equate to improved outcomes. Nishisaki et al. hypothesised that a ‘just in time’ simulation-based intubation refresher, provided at the beginning of each shift, would improve intubation safety in the PICU. The rate of first intubation success and overall success didn’t differ between phases. The rate of resident participation, rather than respiratory therapist, in the intubation increased post intervention (21 % vs 35 %), but the rate of adverse events also increased and was associated with resident participation (OR 2.22) . The Australian evaluation of the PROMPT program was unable to demonstrate any significant changes in clinical outcomes over a 2 year period, in contrast to the UK data . A Californian stepped wedge study of 12 neonatal units introducing a SBE program for neonatal resuscitation for very low birth weight (VLBW) infants between 22–31 weeks gestation showed no difference in the primary outcome of survival without chronic lung disease over 2626 eligible VLBW births .
Improvements in simulated patients with SBE programs
A study of just-in-time training in paediatric sedation showed a significant improvement in the proportion of residents who performed positive pressure ventilation and time to ventilation in response to a simulated apnoea and desaturation during paediatric sedation .
The management of simulated paediatric anaphylaxis has been studied in the operating theatre and the emergency department. In the OT delays in diagnosis and management, including CPR, were common. Only 1/17 participants gave the recommended dose of epinephrine . In the ED study only 50 % of staff administered epinephrine. Epinephrine was more likely to be administered if the scenario included hypotension, suggesting participants felt the diagnosis of anaphylaxis depended on hypotension . An international study of paediatric anaphylaxis in 28 institutions from six countries demonstrated at least one medication error was made in 68 % of simulations, with the most common involving epinephrine .
Improvements in non-technical skills with SBE
Kosok et al. undertook an interesting simulation to raise awareness of biases associated with paediatric sickle cell disease. Biases in communication can harm perceptions of care and alter medical decision making. Biased language was introduced on two occasions by standardised participants in the scenarios, and opportunity given to address this at the time by participants. Participants identified that they didn’t recognise the framing by other members of the care team before it was addressed by the debriefing, that they feel biases every day based on patients’ bed location, frequency of ED visits and wait times. Most participants stated they didn’t know how to correct the biased language .
Simulation for communication skills training is well described in the NICU . A simulation-based study of obstetricians and neonatologists counselling a Standardised Patient pregnant with a foetus at the border of viability found inconsistencies in quoted survival rates, likelihood of survival without impairment, risk of long-term disability, and in the use and interpretation of terms such as “intact survival” between and within the two physician groups. Rehearsal simulation and debriefing immediately before actual antenatal consultations can improve communication techniques, including building rapport and displaying empathy . Families report that empathy, kindness and trust have the most positive impact on their wellbeing and satisfaction with care . Simulation training has also been shown to increase comfort with relaying bad news, such as the death of an infant and exhibiting desired behaviours such as brief pauses and body positioning .
A study of paediatric anaesthetists evaluated the effectiveness of a simulation-based mastery learning curriculum on difficult conversation skills. A pre-post study evaluation was performed with simulated parents. The 2-hour curriculum focused on SPIKES (Situation, Perception, Invitation, Knowledge, Emotion, Summarise) framework for breaking bad news and the NURSE (Naming, Understanding, Respecting, Supporting, Exploring) statements for expressing empathy. Only 15 % of participants met the minimum pass score at pretest, compared to 100 % post-test. It showed significant improvements in specific checklist items of “assesses family’s perception of medical situation before breaking news,” “asks permission before giving the news,” “gives a clear and concise ‘warning shot’,” and “pauses after delivering bad news”. “Ensures family understanding” only showed slight improvement, from 30 % to 47 % .
Using SBE for team training
Regular, weekly, in situ simulation team training of paediatric Medical Emergency Team (MET) staff improves the hospital response to deteriorating patients. The program showed significant improvements where deteriorating patients were recognised more promptly, more often reviewed by consultants, more often transferred to high dependency care, and more rapidly escalated to intensive care. There was reduced PICU admissions, reduced PICU length of stay and reduced PICU mortality. Introduction of the MET program coincided with a significant reduction in hospital mortality. The total annual cost of training (£74,250) was more than offset by savings from reduced PICU bed days (£801,600 per annum). The authors concluded that lessons learnt by ward staff during the regular training led to significantly improved recognition and management of deteriorating in patients with evolving critical illness .
A PICU study focusing on decompensating medical crises (DMCs), events characterised by respiratory and haemodynamic instability secondary to the patient’s underlying illness, showed interdisciplinary SBE achieved immediate acquisition of significantly improved team performance behaviours. The authors felt that because DMCs were not managed based on algorithmic guidelines, resuscitation of these patients required a higher level of team performance and critical thinking .
Ward based paediatric code simulations have been shown to significantly improve team performance over a series of simulations, with one study showing a 3.4 times improvement in team global rating scale, especially team dynamics and communication . In a deteriorating bronchiolitis simulation a significant improvement in team performance was observed with repeated testing after initial simulation and debriefing .
A review on multidisciplinary simulation based trauma team training by McLaughlin et al. demonstrated that simulation based training for trauma resuscitation is associated with improved measures of teamwork, task performance and speed, knowledge and healthcare worker satisfaction . A study comparing didactic lectures to simulation education and measuring outcomes in real trauma patients showed similar knowledge acquisition, but improved teamwork skills in the simulation group . Another study found that implementing simulation based multidisciplinary trauma team training reduced the time to critical operation from 2.6 hrs to 0.6 hrs post intervention, but increased the overall length of stay in ED . Capella et al. showed significant improvements in real patient endpoints with improvements in time to arrival in the CT scanner, endotracheal intubation, and the operating room. They also showed significant improvements in teamwork domains of leadership, situation monitoring, mutual support and communication . Steinemann et al. observed 244 real life blunt trauma resuscitations 6 months before and after the introduction of a four-hour simulation based insitu interdisciplinary program. They showed significant improvement in teamwork scores along the time frame, with objective improvements in speed and completeness of tasks. This was illustrated by a 76 % increase in near perfect task completion and a reduction in mean overall ED resuscitation time by 16 % .
Successful teamwork is essential for good patient outcomes and attributes of a successful team include psychological safety, member dependability, role structure and clarity, meaning of work and impact of work. Psychological safety is the most important factor and is a shared belief that the team is safe for interpersonal risk taking, including asking for help, admitting errors, and seeking feedback without fear of retribution . Pollack et al. have shown in eight American NICUs that better organisational and managerial practices were associated with improved patient outcomes including lower 28-day mortality. Scores for leadership, coordination and conflict resolution were particularly important . An evaluation of an interdisciplinary simulation program with a specific focus on highlighting psychological safety yielded interesting results. Residents and nurses both rated their psychological safety low, pre and post intervention, with residents’ ratings lower than nurses. Qualitative analysis identified six themes [ Table 2 ] . There is also evidence that executive functions, mindfulness, and stress influence the behaviour and skills of team members in paediatric emergencies. Components of mindfulness, such as non-judgment and conscious action, were positively related to the skills of medical team leaders. Executive functions correlated with the non-technical skills and mindfulness .
| 1. Influence of existing relationships on future interactions |
| 2. Unsatisfactory manner and frequency of communication |
| 3. Unsatisfactory resolution of disagreements |
| 4. Overwhelming resident workload impairs collaboration |
| 5. Interpersonal disrespect disrupts teamwork |
| 6. Interprofessional simulation was useful but not sufficient for culture improvement |
A study in paediatric anaesthesia measured participants personality and decision-making style and compared this to their performance in a simulated hypoxic cardiac arrest. They showed that adequate treatment was inversely correlated to avoidant and spontaneous decision-making styles. Both styles are likely to generate inadequate responses to these specific situations due to the absence or the inadequacy of responses. Outcomes were positively correlated to the agreeableness and conscientiousness personality traits. Consequently, both agreeableness which promotes team coordination − and conscientiousness − which promotes planned organisation and discipline in applying adequate measures − appear as the ideal combination insuring successful management of acute critical conditions .
Young et al. examined the workload of learners in different team roles during simulated paediatric CPR. Task load was measured using the NASA Task Load Index which provides an average workload score across six domains: mental demand, physical demand, temporal demand, performance, frustration, and mental effort. The team leader reported the highest workload, while the airway reported the lowest. The team leader had higher scores for all sub scores except physical demand. The team leader reported the highest mental demand, while airway reported the lowest. Cardiopulmonary resuscitation coach and first responder reported the highest physical demands, while team lead and nurse recorder reported the lowest . The same NASA Task Load Index was used by Tofil et al. to examine workload during a simulated paediatric sepsis scenario. The team leader had significantly higher overall workload than team members. Team leader had higher workloads in all subcategories except in performance where the values were equal and in physical demand where team members were higher than team leaders. The highest category for each group was mental for team leaders and team members. For the team leader, two categories, mental and effort, were high workload, most domains for team member were moderate workload levels . Both these studies demonstrate the need for a team leader to delegate some tasks to other team members to reduce their workload.
Simulation as a tool in Quality and Safety
A Canadian study examined adherence to paediatric cardiac arrest guidelines across fifty EDs using in-situ simulation. The study demonstrated variable adherence across the spectrum of EDs, which wasn’t associated with paediatric volume. Improved performance was not directly related to simulation teamwork performance, or more providers with PLS training . An American study examining the quality of care provided during simulated paediatric cardiac arrests showed 66 % of residents failed to start compressions within 1 min of pulselessness and 33 % never started CPR. Only 54 % of residents defibrillated the patient within 3 min of pulseless ventricular tachycardia .
Simulation can test the use of guidelines and cognitive aids. Hall et al. compared use of a cognitive aid vs. no aid and demonstrated significant reductions in error rates for simulated cases of newborn resuscitation (mean error rate 14.5 % vs. 37.3 %) and paediatric seizures (12 % vs 33.3 %) .
Monitoring paediatric resuscitation simulations enable the identification of drug preparation and administration errors. Medication errors during emergencies are 39 times more likely to lead to harm and 51 times more likely to result in death when compared to non-emergency care . Murugan et al. examined 96 dosages in simulated resuscitation. 13/96 had > 20 % withdrawal phase error, 20/96 had > 20 % administration phase error. Ketamine had the highest withdrawal errors, epinephrine the highest dilution and administration errors . Appelbaum et al. observed an error rate of 52/180 (29 %) of medications administered. 22 of these were moderate or severe severity. Only two errors were noticed by staff after administration and therefore may have been reported in clinical practice . A study of prehospital simulated anaphylaxis demonstrated 20 % of crews gave > 5 times the correct dose of epinephrine, 14 % gave the dose of epinephrine intravenously rather than intramuscularly. Causes of errors were categorised as faulty reasoning, weight estimation errors, faulty recall of medication dosages, problematic references, calculation errors, dose estimation, communication errors and medication delivery errors .
Alsaedi et al. evaluated the simulated resuscitation of a neonate with coarctation of the aorta presenting in cardiogenic shock across 12 general EDs. The primary outcome was adherence to best practice, as measured by a 15-item overall composite adherence score (CAS). Only 28 % of teams reached the correct diagnosis. The three most underperformed tasks were performing focused exams such as pulses on the upper and lower extremities, obtaining blood pressure in the upper and lower extremities separately and administering prostaglandin E1 infusion using the correct dose. The authors concluded that these gaps in diagnosis and/or management were mainly congenital heart disease critical items and highlight the importance of a targeted improvement program to enhance the quality of care provided for such a high-stake, low frequency illness in the general ED .
Lauridsen et al. used a simulated teenage PEA arrest to validate the development and introduction of a standardised communication tool to be used before pausing compressions, a countdown, and action words for pausing and resuming compressions. Compared to control group the new standardised communication tool resulted in significant reductions in perishock pause, intubation pause and rhythm check pause. They also showed significant reductions in median frustration index and median mental demand load .
The PEARLS debriefing tool has been adapted for debriefing system focused simulations. Just as clinical care can be simulated, so can the patient journey. Separating out the care and simulating the other parts of the process can identify system failures and inefficiencies, as well as testing new processes before they go live. System focused simulation recreates complex systems of care and seeks to understand system issues. It conducts simulations insitu, using real equipment and fully implementing processes (e.g. Real Code Blue team activation) .
Paediatric in situ simulations in paediatric emergency departments have been used to identify latent safety issues and incidents of suboptimal care. Lee et al. found deficiencies in the immediate availability of paediatric specific equipment as well as lack of a cognitive aid for reference ranges for normal paediatric vital signs. This study also showed improvements in Paediatric Readiness Scores after 12 months . An Australian study identified 194 incidents of suboptimal care attributed to 325 causation factors. There were 76 knowledge deficits, 39 clinical skill deficits, 36 leadership problems, 84 communication failures, 20 poor resource utilisations, 23 preparation and planning failures and 47 incidents of a loss of situational awareness. Clinically important themes included paediatric life support, drug choice and doses, advanced airway and ventilation, intravenous fluids and recognition of the deteriorating patient. Recurring incidents included the failure to recognise a cardiac arrest, inadequate fluid resuscitation and incorrect medication dose administration .
Simulation in high stakes assessment
Competency based medical education (CBME) uses an outcomes approach to the design, implementation, assessment, and evaluation of a program. Knowledge, skills, and behaviours need to be assessed to measure competency. Competence evolves from novice to mastery through a career . Many methods can be used to measure competency and include checklists, rating scales, behaviourally anchored rating scales (BARS), global rating scales (GRS), cumulative sum analysis (CUSUM). Assessment tools should be valid and reliable. There are generally five validity domains: content, response process, internal structure, relationship to other variables and the consequences.
When considering using simulation as an assessment rather than an education tool, one must remember Miller’s Pyramid ( Fig. 2 ). Participants do not necessarily behave in simulation as they would in real life, and simulations do not necessarily represent real life. The emphasis must be on assessment organisations to rigorously develop and evaluate their assessment tools, such as that created for ED residents by Hall et al. . They highlighted an important point where with essential item checklists less experienced residents sometimes performed better than the more experienced. This is because those who have reached mastery may not follow a step wise approach, but will do better overall clinically and receive superior GRS scores . Weersink et al. used an anchored GRS to generate an entrustment score and compared emergency medicine residents performance in simulation based examinations to the ED during resuscitations, showing a moderate positive correlation between scores (r = 0.630, p < 0.01) .
