© Springer-Verlag London 2015
Paul R. Barach, Jeffery P. Jacobs, Steven E. Lipshultz and Peter C. Laussen (eds.)Pediatric and Congenital Cardiac Care10.1007/978-1-4471-6566-8_77. Professional Formation of Physicians Focused on Improving Care
(1)
Department of Field Activities, Accreditation Council for Graduate Medical Education, 515 North State Street, Suite 2000, Chicago, IL 60654, USA
(2)
Quality Improvement and Assessment Programs in Pediatric Practice, American Board of Pediatrics, 12 Wedgewood Rd, Chapel Hill, NC 27514, USA
Abstract
There is evidence that many patients do not receive optimum care despite efforts to improve health care quality. Society expects physicians to improve care for their patients, and to lead and function as members of quality improvement (QI) teams. A range of educational approaches exist to prepare physicians for this important role. Education in QI across the educational continuum is supported by the requirements of educational accrediting organizations and the expectation of specialty certification board. These regulatory bodies expect that physicians develop these skills during training, and that physicians in practice maintain QI competence, and assess and improve their patients’ care. At the same time, no coordinated curriculum for teaching QI across the continuum of medical education exists to date. An effective approach to QI education encompasses both didactics and immersive experiences that enable learners to apply their developing competence to real-world problems. Given the importance of team-based approaches in the care of complex patients, new multidisciplinary QI approaches, informed by research on what makes care effective, will contribute to care that improves the patient experience. These will be supported by advances in medical training and assessment, healthy populations, and will lead to improved quality and lower per capita cost of health care to benefit patients and society.
Keywords
Medical educationProfessional developmentPatient safetyQuality improvementEducational continuumMulti-specialty and team-based careTriple aimPractice-based learning and improvementSystems-based practiceThe Quality Imperative
Quality and safety of care are important in all healthcare settings. Assuring safe care is particularly challenging for critically ill patients, complex care, human-technology interactions, and care by multiple health professionals and teams. Donabedian provided a framework for assessing and addressing quality problems in healthcare in a groundbreaking book published in 1966 [1]. Despite innovative work and critical thinking over more than four decades to develop and implement quality improvement (QI) programs, studies still show significant variations in the quality, safety, utilization and cost of healthcare services [2]. Children, for example, receive only 46 % of recommended services [3], and a study using national data for Medicare beneficiaries found that increased use of specialists and larger healthcare expenditures did not always translate into better outcomes and at times increased pain and suffering [4].
Crossing the Quality Chasm, a 2001 report by the Institute of Medicine of Medicine (IOM) asserted that gaps in quality of care result in part from health professions education that has not kept pace with new team-based approaches to care, use of technology and informatics, and understanding patient expectations for their experience of care [5]. To address these concerns, the IOM put forth six aims for the healthcare system: care should be effective, safe, patient-centered, timely, efficient, and equitable [5]. The IOM’s seminal report on patient safety, To Err is Human, also noted that besides problems with the design of the care system, educating health professionals about avoiding errors and adverse events is critical to safe and effective healthcare, but that these concepts are not included in the educational curricula in most health professions [6].
Research has shown that quality and safety of care are inextricably linked to the education and ongoing professional development of physicians, and that the quality of formal training has a long-term and profound effect on the quality of care a physician delivers over a lifetime of practice. A study of the effect of training on obstetrician-gynecologists’ performance in practice showed that the complication rates for patients cared for by physicians who had trained in programs who ranked in the bottom quintile for risk-standardized major maternal complications was one-third higher than the rate for patients cared for by graduates of programs in the top quintile [7]. Enhancing formal education and continuing professional development related to QI and improvement science is essential to close these gaps. The aim of these programs is to develop physicians who are lifelong learners who can continuously self-assess and improve their performance and outcomes. Needed skills include a comprehensive understanding of QI, the ability to reflect on the patient and the processes of care, and the skills to apply these principles in day-to-day practice to enhance quality and avoid medical errors and preventable adverse events. The Dartmouth Health Atlas’ website, citing long term influence of the training program on the quality of care physicians deliver in practice, emphasizes that residents should know the quality of care delivered by institutions and clinical departments as a key factor in selecting their residency program [8].
While the earliest professional group with involvement in quality improvement is nursing, going back to Florence Nightingale during the Crimean War [9], the history of physician involvement in efforts to improve the quality of care traces back to Ernest Codman and his efforts to improve surgical care at the Massachusetts General Hospital in the early twentieth century [10]. Codman developed a system to assess where problems with the quality of care originated, address the causes [11], and use these principles to inform a set of standards for hospitals that over time evolved into the Joint Commission for the Accreditation of Healthcare Organization (now TJC) [10]. Quality improvement in industrial production began in the 1940s, through the work of Shewhart, Deming and others [12]. These concepts did not become widely disseminated and applied in healthcare until the late 1980s [13]. Prior to the adoption of QI concepts from industry, efforts to manage quality in healthcare largely were separate from medicine and medical education, and focused on meeting regulatory and accreditation requirements, compliance and utilization review [14]. Once QI principles were introduced in the healthcare sector, another decade passed before Headrick, Batalden, Leach, Davidoff, Berwick and others introduced these concepts into medical education [15]. Out of this work emerged organizing principles for quality improvement that include: (1) understanding healthcare as processes within a system; (2) understanding variation in care and the need to measure it; (3) knowing the effect of illness and healthcare on patients; (4) understanding the process of making changes in healthcare and the roles of leading and participating in these efforts; (5) collaborating in teams and groups; (6) dealing with social context and accountability; and (7) developing and applying locally useful knowledge [16].
Following the publication of the IOM’s “To Err is Human” [6] and “Crossing the Quality Chasm” [5] reports, the IOM addressed the needs for education and professional development of health professionals related to QI in a third report, entitled Health Professions Education: A Bridge to Quality [17]. Collectively, the three IOM reports have contributed to more widespread teaching of quality and patient safety concepts across the continuum of medical education [18], and the education of nurses [19] and other health professionals [20].
This chapter discusses the components of the professional formation of physicians related to improving the quality and safety of patient care across the continuum from undergraduate and graduate education to continuing professional development. It also describes the barriers and facilitators to QI learning, and the assessment of educational and clinical outcomes related to improvement learning. Practical examples of physicians’ professional formation in quality and safety improvement are presented throughout. The chapter concludes with a research agenda to promote physicians’ professional development relevant to quality and safety.
Components of Health Professional Formation Related to Improvement
The IOM’s 2003 report Health Professions Education: A Bridge to Quality identified five core competencies to meet the needs of the twenty-first century healthcare system, which span the various professions involved in healthcare [17]. The IOM competencies have been used to define goals and objectives for professional formation and development across a range of professions. They build on earlier efforts to define common competencies across the different health professions [21–23], and a growing recognition that much of the work to improve healthcare is done in interprofessional teams.
The aim of educating physicians in quality improvement is to ensure they develop skills and competencies that enable them to improve care in their local clinical context, as well as in the larger system of care. Knowledge of QI science is relevant, yet equally important is the ability to apply these tools in a real-world, organizational context. Added competencies particularly relevant to the implementation components of QI work include communication, teamwork, analytic skills and an understanding of healthcare systems. Crossing the Quality Chasm also identified commitment to lifelong learning as an important competency [5]. Lifelong learning consists of formal education and ongoing professional development that is required for physicians and other licensed professionals, along with learning through self-directed studies, reflection and interaction with peers.
Recognition that the predominant focus on medical knowledge and clinical skills underemphasized other areas relevant to the delivery of high-quality care prompted the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Medical Specialties (ABMS) to develop a broader approach to defining the skills and attributes that physicians should have to deliver quality are. At its core are six competencies relevant to the work of physicians: (1) patient care, (2) medical knowledge, (3) interpersonal and communication skills, (4) professionalism, (5) practice-based learning and improvement (PBLI), and (6) system-based practice (SBP). In 1999, these competencies were formally endorsed by the ACGME for the education and assessment of physicians in training [24], and by the American Board of Medical Specialties (ABMS) for the certification and ongoing professional development of physicians in practice [25]. While all ACGME/ABMS competencies are relevant to the delivery of high-quality care, PBLI and SBP – are particularly pertinent to the role physicians need to play in ensuring quality of care. There is considerable conceptual overlap between the ABMS/ACGME core competencies aimed at individual physicians, and the IOM competencies for a safer and more effective healthcare system.
PBLI encompasses the process physicians use to assess and improve their knowledge and skills relevant to their practice, including development of learning and improvement goals, initiating professional development activities, and evaluating the outcomes in a self-guided approach [26]. Improvement often begins with the identification of a clinical area or the care of a specific group of patients, with a focus on tailoring interventions to address gaps in quality and safety identified via self-assessment or external information such as clinical outcomes or patient satisfaction. Resources for this process include new medical evidence, and there is conceptual overlap between PBLI and evidence-based medicine [27].
Embedded in SBP is an understanding of the systems of care, along with participation in efforts to improve them. Activities include advocating for high-quality patient care; working in interprofessional teams to enhance the quality and safety of services; and participating in identifying and diagnosing system errors and implementing solutions.
There have been a number of attempts to define quality and safety curricula, including, among other, an annotated bibliography on teaching quality and safety during formal medical education [28], a patient safety program for physicians in practice developed [29], and a curriculum for residents in surgical specialties [30]. Recently, the Institute of Healthcare Improvement’s (IHI’s) Open School has made publicly available a broad range of curricular materials relevant to healthcare quality and safety, and these resources are being used in the education of residents and medical students, and in nursing and other health professions [31, 32]. While there are differences across curricula, and no universally accepted quality and safety curriculum exists to date, there is considerable overlap across professions and different formulations.
The Educational Continuum
A model of education that encompasses all forms of learning, from formal education and instruction to the ongoing learning that continues throughout an individual’s professional career is important for a concept of professional formation [33]. For physicians, this encompasses undergraduate medical education, residency and fellowship training, and ongoing professional development for physicians in practice.
Undergraduate Medical Education
A century after Codman’s pioneering work to introduce outcome based medicine into healthcare and 30 years after the introduction of quality improvement, it is gratifying that more medical schools are including these concepts in their curricula and approaches for teaching and assessing students [34]. Ideally, these concepts are taught at the beginning of student clinical experiences and become an integral part of the ongoing development of physicians’ clinical improvement skills [34]. Some medical schools have implemented focused QI electives for students in the clinical years [35]. The World Health Organization (WHO) emphasizes that patient safety and QI should be taught to medical students [36]. In contrast, the standards of the Liaison Committee on Medical Education, which accredits US and Canadian medical schools, do not yet include formally mention curricular elements or require competence in healthcare quality and safety [37]. Recommendations from the National Patient Safety Foundation (NPSF) on enhancing patient safety call for medical schools to emphasize the ACGME/ABMS competencies, patient-centered care, and working in interdisciplinary teams [38]. The NPSF also has recommended that teaching about patient safety should begin on the first day of medical school and should encompass the science of error causation and mitigation, the study of human factors, safety improvement science, systems theory and analysis, and systems design [38].
Graduate Medical Education
Graduate medical education is the phase of the educational continuum during residency and fellowship training where much of physicians’ education about healthcare quality and safety occurs. While many of the first QI curricula targeted medical students, a few were aimed at both students and residents. In 2002, recommendations called for a broader approach to teaching QI to physicians, including expanded curricula, a focus on creating an organizational culture conducive to improvement and assessing outcomes at the individual and program levels [39]. Early efforts focused on non-mandatory “QI electives” [40], while more recent implementation of QI in the residency curriculum entails mandatory QI rotations and experiences for all residents [41–43].
Some teaching efforts have used active engagement in projects that integrate didactics with the application of what is learned [44]. This is based on the principles of experiential learning [45, 46]. The underlying theory is that teaching should be grounded in learners’ experiences and that these experiences themselves represent a valuable educational resource [46]. Learning by doing and “just in time training,” are related approaches that have been found useful in teaching QI principles and practices to learners across a range of health professions [47]. Efforts have also focused on the creation of templates that guide learners through the QI process, and that allow QI efforts to be recorded like other scholarly projects [48].
Currently, five approaches are commonly used for teaching residents PBLI, SBP and related concepts: (1) didactic sessions, including lectures, case studies, journal clubs and on-line self-study modules; (2) incorporating QI principles and concepts into clinical events such as morbidity and mortality (M&M) and clinical case conferences, and morning report [49–51]; (3) resident-initiated quality improvement projects [52, 53]; (4) resident participation in program-level QI projects; and (5) participation in multi-disciplinary institutional QI efforts [54]. Readily implementable modules, some of which are online, are being used to facilitate improvement in outcomes for common diagnoses such as diabetes and asthma are being used particularly in ambulatory and primary care settings [55–57]. Some improvement efforts have a longer term focus, multiple PDSA cycles, and include a focus on spread and sustainability of improvement [58]. A matrix that crosslinks the Institute of Medicine (IOM) aims with the ACGME competencies has also been used to make QI concepts meaningful to residents [59]. Efforts to involve residents in larger department or institutional QI efforts are less common. Some examples include addressing waiting times in pediatric urgent care [60]; improving patient flow in a resident continuity clinic [40]; use of practice data to enhance compliance with immunization guidelines [61]; standard order sets for patients with community-acquired pneumonia [62]; and a resident-led initiative to improve communication between the inpatient medical staff and ambulatory physicians [63]. The AAMC’s Teaching for Quality report offers broad recommendations for medical schools and teaching hospitals to incorporate the principles and practice of Quality Improvement and Patient Safety (QI/PS), including learning experiences and use of these concepts in the assessment of physicians across the continuum of education and practice [64].
Continuing Professional Development for Physicians in Practice
Since the 1970s, medicine, nursing and several other health professions require continuing education to maintain a professional license [65]. To ensure the effectiveness and quality of continuing education for physicians, providers of continuing medical education must be accredited by the Accreditation Council for Continuing Medical Education (ACCME) [66]. A review of the literature on the effectiveness of continuing medical education (CME) found a positive effect on attitude and practice behaviors, and some improvement in outcomes [67]. At the same time, the value of traditional CME is being challenged, with critics noting that CME is mostly delivered in lectures, with little post-participation assessment beyond learner reactions and self-reported changes in behaviors [68, 69]. In contrast, interactive learning, and participating in multiple activities focused on the same topic are effective in producing moderate to significant change in participants’ behavior [70, 71]. A 2010 IOM report on redesigning continuing education in the health professions noted flaws in existing approaches that include narrow curricula, a failure to focus education on individual and collective knowledge and capability gaps [72], and gaps related to quality and safety of care.
The responsibility for assessing ongoing professional development of physicians in practice lies with the member boards of the American Board of Medical Specialties (ABMS). Board certification grew out of concerns about the quality of physician education and preparation for practice at the turn of the twentieth century, resulting in the establishment of the Advisory Board of Medical Specialties, the precursor of the American Board of Medical Specialties in 1933 [73].
Initially, board certification was achieved by passing a secure written and for some specialties and oral examination at the end of formal training and was valid over a physicians’ entire career. Starting in the 1970s, several ABMS boards, acknowledging that medical knowledge changes rapidly over the course of a physician’s career initiated a process of periodic re-certification through repeat secure knowledge examinations. Over time all member boards gradually moved from life-time certificates to time-limited certificates and periodic repeat examinations [74, 75]. The American Board of Family Medicine has required periodic recertification since its inauguration in 1969 [76]. Research on the effect of board certification has found a positive and statistically significant association between specialty board certification and greater compliance with recommended treatments and improved outcomes. Certification for internists is associated with improved patient care [77], and prevention of medical errors [78]. Examples from the surgical field include lower mortality and complication rates for surgical procedures, including carotid endarterectomies and aortic aneurysm surgery [79], colon surgery [80], and surgery for peptic ulcers [81].
In 1999, prompted by documented gaps in quality and safety of care in almost all areas of medicine and a growing focus on improving care, the ABMS redesigned the certification process to ensure ongoing competence by including expectations that all diplomates engage in measuring and improving the quality of care they provide, using the framework of the six competencies first developed by the ACGME and adopted by ABMS [73]. Over the past decade, certifying boards have moved from assessing professional development primarily based repeated assessment of medical knowledge through a secure examinations and participation in CME to use of self- and peer assessments of other competencies such as communication, professionalism, actual delivery of care, the ability to function in complex systems and the ability to assess and improve quality of care in practice [74]. The resulting four part Maintenance of Certification® (MOC) framework became official ABMS policy in 2000 and by 2006 all 24 member specialty boards had established time lines for implementing MOC [74].
The MOC has four components. Part I requires physicians maintain an unrestricted medical license [74], and surgeons must document they have privileges or a staff appointment in a hospital accredited by the Joint Commission [82]. Part II requires completion of a specified amount of CME or acquisition of specialty specific medical knowledge, some of which must relate to the physician’s clinical practice [83]. Part III, “Cognitive Expertise,” consist of a completing a periodic secure specialty examination, and Part IV, “Performance in Practice,” requires physicians to assess their practice performance, with some boards requiring improvement data for a sample of patients, such as data extracted from medical records and clinical databases, or results of patient surveys [74]. Physicians may use Web-based improvement modules, such as the Patient Safety Improvement Program developed by the ABMS [84], or the improvement modules developed by the American Board of Pediatrics, American Board of Family Medicine and the American Board of Internal Medicine. Physicians are encouraged to aggregate their performance data into a portfolio that documents how their diagnostic and clinical decisions and outcomes compare to those of peers and available national comparisons. For surgeons, the MOC Part IV offers credit for ongoing participation in a national, regional or local outcomes registry or quality assessment program, and the American Board of Surgery requires that this improvement program address areas specific to the individual physician’s practice [82]. The concept of continuous professional development integrated with PBLI is considered important in helping surgeons improve the care they provide to patients [85].
Physicians are encouraged to participate in proven QI efforts that have been shown to improve care to address gaps in quality in areas in need of improvement. Part IV activities in several specialties use Practice Improvement Modules (PIMs), which facilitates collaborative, QI efforts within practices and across practices [86, 87]. An added advantage of PIMs is that they can be used in teaching settings to collect longitudinal improvement data for individual residents [86, 87]. Several of the MOC programs also award Part IV MOC credit for diplomate participation in approved ongoing QI efforts in their practice setting. Some of these efforts are multi-center prospective ongoing QI networks that have demonstrated significant improvement in outcomes of care and at times reduction in cost of care [88]. Other examples include medication reconciliation [89], prevention of childhood obesity [90], and prevention and management of cardiac disease [91], among others. Some boards have made significant gains in introducing improvement activities into the professional development of practicing physicians while for other specialties MOC is still early in its implementation. Fifteen of the ABMS boards have agreed to common standards for awarding credit for MOC Part IV to diplomates who participate in organizationally sponsored QI efforts through the Multi-Specialty Portfolio Program [92]. Data on the effectiveness of MOC in improving the safety and quality of care is growing and can be tracked on the ABMS website (http://www.ABMS.org).
Physicians’ role in institutional healthcare improvement requires a new skill set for individuals who function as institutional leaders in quality and safety, with a formal description of the role and its responsibilities [93], and the requisite knowledge and training, including implementation science and policy work [94, 95]. Examples of programs for training these quality experts are found in the Department of Veterans Affairs [96], Dartmouth College [97], and George Mason University [98].
Interprofessional and Team Learning
Given the variability in how and to what degree curricula and educational approaches in health professions education emphasize improving quality and patient safety, one solution that has been proposed is interprofessional education, which has been generally found to enhance quality and reduce cost in healthcare [99]. The value of interprofessional education has been emphasized since the 1970s [100], and beginning in 2013, the LCME standards require medical schools to prepare students to function collaboratively on teams that include other health professionals in training and practice [101].
The “Retooling for Quality and Safety” initiative of the Josiah Macy Jr. Foundation and the Institute for Healthcare Improvement has developed a curriculum focused on interprofessional training, involving both medical and nursing students [102]. The initiative fostered integration of improvement and patient safety curricula in undergraduate medical and nursing education, emphasizing the value of interprofessional learning because the didactic curriculum is taught the same way actual improvement occurs, in interprofessional teams addressing quality problems in a real clinical setting. The advantage of this approach is that “learning how to do quality improvement and actually carrying out quality improvement are essentially one and the same; both are special forms of experiential learning” [103]. A few successful models exist for the pre-licensure phase of education [104], and there is a need to expand this work to the continuum of health professions formation.
Curricula for interprofessional learning have included train-the-trainer exercises that facilitate hands-on training [105]. For patient safety, curricular components have included patient safety basics, developing academic leadership, improving the culture of practice, changing the response to error, and applying principles of interprofessional teaching and learning [106]. Interprofessional learning is also being proposed for continuing education in the health professions. A 2010 IOM report on the redesign of continuing education recommended that it be carried out in interprofessional teams that mirror the team composition in the healthcare setting [107]. A recommendation for advancing team-based learning and practice also has called for the development of community learning sites to serve as venues for interprofessional learning and practice [108]. This type of interprofessional learning and practice does not yet occur in many settings, although there are efforts to conduct e-learning activities that provide a matched curriculum to multiple health professions, to promote interprofessional learning [109]. In addition the IHI Open School modules are suitable for use across difference health professions [31], and many of the quality improvement efforts approved for MOC Part 4 credit involve interdisciplinary care teams [74].
Barriers and Facilitators to QI Learning
Some of the barriers that account for the slow spread of QI in health professions education include a shortage of faculty able and prepared to teach these concepts in the clinical setting, discipline-specific educational “silos” that conflict with the need to teach and practice quality and safety improvement in multi-professional and inter-professional teams, perceptions that physicians in training must first acquire “clinical skills” before engaging in quality improvement, and a shortage of time and opportunity to insert hands-on experiences in quality and safety into packed medical education curricula. A final barrier to learner involvement in QI is the availability of real time patient quality and safety data related to the populations served by residency and fellowship programs. Even in organizations with well-developed quality improvement efforts, capabilities for assessing clinical outcomes of care in training programs lag behind most other areas.
The most frequently cited barrier is the lack of clinically based faculty members with the skills to teach improvement of care and the effect this has on the “informal curriculum” at the bedside and in the clinic. For example, the Macy Foundation’s effort to promote interprofessional education in quality and safety found that a common challenge was the lack of sufficient faculty across all health professions trained in the principles and processes of improvement [102]. Teaching of quality and safety thus far has relied to a large degree on a small core of regional and national experts presenting lectures and educational modules. Many frontline teachers, not familiar or comfortable with quality and safety principles fail to enforce these concepts in the clinical context in training programs. Learning is influenced by organizational culture and the “informal curriculum” that can undermine teaching and valuing quality and safety especially when the faculty physicians are not engaged in meaningful quality improvement and safety efforts in their own practices. The informal curriculum is important because it teaches the “values of the profession,” [110] and deeply influences professional formation [111]. Through this, it can enhance or thwart formal efforts to introduce new concepts, including added emphasis on approaches to enhance the quality and safety of care [110, 112, 113]. Work to overcome the informal curriculum as a barrier to meaningful, effective QI exposure for residents has focused on faculty development to improve their bedside QI teaching [64, 113], and enhancing faculty understanding of the power of the informal curriculum.
Recommendations for overcoming these barriers have focused on the alignment between the formal and informal curriculum to contribute to a learning healthcare system, including appropriate role modeling of QI practice by teaching faculty [113]. The AAMC’s recommendations for integrating quality improvement and patient safety into faculty competencies include a focus on faculty development in these areas [64]. Practical efforts congruent with these principles have focused on QI becoming a more integral part of the residency curriculum by coordinating teaching of PBLI and SBP with resident engagement in improvement activities. This gives residents the opportunity to apply their new learning in a local context and to contribute to improving care. Active involvement of learners was recommended as early as the mid-1990s [114]. Nearly two decades later, models to achieve this are being developed and tested, with a focus on five components of a meaningful QI experience for leaners: (1) curricula and education models that ground learners in the principles of QI; (2) faculty preparation for teaching QI and practicing it in clinical settings; (3) ensuring all learners receive QI education; and (4) overcoming time and other constraints to allow them to apply newly developed QI skills, and (5) assessing the effect of exposure to QI on learners’ competence [115].
The final barriers to effective QI education for physicians are the constraints on available time posed by packed clinical curricula and the time intensive nature of a meaningful QI immersion experience. This makes it challenging to meet expectations that all learners should be exposed in a meaningful way to QI, and have the opportunity for active participation in efforts to improve care. Efforts to overcome this barrier include revisions to the scheduling of residents’ QI experiences. At one institution this resulted in a change of the QI experience first from a 1-month experience at 100 % time, to a 3-month experience at 50 % of the residents’ time and ultimately to 9 months at 15 % protected time for QI to allow residents to initiate or participate in longitudinal improvement projects [116]. QI immersion programs for residents in several institutions have shown that it is possible to achieve meaningful resident involvement in QI within the established duty hour limits for medical residents [42, 117].
Assessing Educational and Clinical Outcomes of QI Education
Established methods of evaluation, such as examinations that test the acquisition of knowledge are not optimally designed competence in an applied concept like QI, which both requires experiential learning and application of the concepts in practice [118]. Assessment has focused on a broader set of learning outcomes such as gains in knowledge, changes in attitude, acquisition of skills and ability to engage in QI activities, actual clinical outcomes, and whether an intervention produced other benefits for patients or the healthcare system.
The Kirkpatrick framework for assessing the outcomes of learning is the approach most commonly used, and has been the basic model for assessing educational and training interventions across a range of industries [119]. Developed more than 50 years ago, it offers a comprehensive approach that assesses adult learning, taking into consideration the needs of the learner, the instructor, the larger system and the stakeholders, such as patients and the larger healthcare system, that the adult learning program is intended to benefit [119]. Kirkpatrick emphasized that the assessment of training should go beyond obtaining information in immediate reactions of the attendees, and assessment should be carried out on four different levels [119]. The four levels consist of: (1) Reaction – How well did the learners like the learning process; (2) Learning – What did they learn? (gains in knowledge and skills); (3) Behavior – (What changes in job performance resulted from the learning process? (gains in capability to perform the newly learned skills while on the job); and (4) Results – What are the tangible results of the learning process in terms of reduced cost, improved quality or efficiency [119].
Research on the outcomes of medical education has consistently found that involvement in QI benefitted residents’ PBLI competence, including skill in designing projects and conducting plan-do-study-act cycles as well as their self-ratings of knowledge and efficacy related to QI [120]. Activities such as improvement exercises, multidisciplinary rounds, chart audits, and opportunities to compare residents’ patient outcomes to relevant benchmarks were found to enhance proficiency in PBLI and SBP [121, 122]. The limitations of the literature on learning outcomes of residents’ exposure to QI principles include the small samples for most studies, the narrow scope and brief follow-up period for many interventions, and the fact that outcomes often are limited to Kirkpatrick Levels 1 or 2. Studies that assessed the outcomes of training in general have found that level 3 is rarely attained and Level 4 is never assessed [123]. A systematic review of patient feedback in improving physician consult behavior found four studies that assessed interventions at Level 4 [124], and another review showed that most QI curricula in the literature resulted in improvement in learner knowledge or confidence to perform QI, yet only a few studies offered evidence that QI education had an actual impact on meaningful behavior change, as well as clinical outcomes [125]. A small number of studies have assessed the effect of a safety or quality curriculum to Level 3 of the Kirkpatrick framework [126].
< div class='tao-gold-member'>
Only gold members can continue reading. Log In or Register a > to continue