Hospital Medicine

ABSTRACT

Objective: Promoting a culture of safety is a critical component of improving health care quality. Recognizing staff who stop the line for safety can positively impact the growth of a culture of safety. The purpose of this initiative was to demonstrate to staff the importance of speaking up for safety and being acknowledged for doing so.

Methods: Following a review of the literature on safety awards programs and their role in promoting a culture of safety in health care covering the period 2017 to 2020, a formal process was developed and implemented to disseminate safety awards to employees.

Results: During the initial 18 months of the initiative, a total of 59 awards were presented. The awards were well received by the recipients and other staff members. Within this period, adjustments were made to enhance the scope and reach of the program.

Conclusion: Recognizing staff behaviors that support a culture of safety is important for improving health care quality and employee engagement. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Keywords: patient safety, culture of safety, incident reporting, near miss.

A key aspect of improving health care quality is promoting and sustaining a culture of safety in the workplace. Improving the quality of health care services and systems involves making informed choices regarding the types of strategies to implement.¹ An essential aspect of supporting a safety culture is safety-event reporting. To approach the goal of zero harm, all safety events, whether they result in actual harm or are considered near misses, need to be reported. Near-miss events are errors that occur while care is being provided but are detected and corrected before harm reaches the patient.^1-3 Near-miss reporting plays a critical role in helping to identify and correct weaknesses in health care delivery systems and processes.⁴ However, evidence shows that there are a multitude of barriers to the reporting of near-miss events, such as fear of punitive actions, additional workload, unsupportive work environments, a culture with poor psychological safety, knowledge deficit, and lack of recognition of staff who do report near misses.^4-11

According to The Joint Commission (TJC), acknowledging health care team members who recognize and report unsafe conditions that provide insight for improving patient safety is a key method for promoting the reporting of near-miss events.⁶ As a result, some health care organizations and patient safety agencies have started to institute some form of recognition for their employees in the realm of safety.^8-10 The Pennsylvania Patient Safety Authority offers exceptional guidance for creating a safety awards program to promote a culture of safety.¹² Furthermore, TJC supports recognizing individuals and health care teams who identify and report near misses, or who have suggestions for initiatives to promote patient safety, with “good catch” awards. Individuals or teams working to promote and sustain a culture of safety should be recognized for their efforts. Acknowledging “good catches” to reward the identification, communication, and resolution of safety issues is an effective strategy for improving patient safety and health care quality.^6,8

This quality improvement (QI) initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. If health care organizations want staff to be motivated to report near misses and improve safety and health care quality, the culture needs to shift from focusing on blame to incentivizing individuals and teams to speak up when they have concerns.^8-10 Although deciding which safety actions are worthy of recognition can be challenging, recognizing all safe acts, regardless of how big or small they are perceived to be, is important. This QI initiative aimed to establish a tiered approach to recognize staff members for various categories of safety acts.

METHODS

A review of the literature from January 2017 to May 2020 for peer-reviewed publications regarding how other organizations implemented safety award programs to promote a culture of safety resulted in a dearth of evidence. This prompted us at the Veterans Affairs Connecticut Healthcare System to develop and implement a formal program to disseminate safety awards to employees.

Program Launch and Promotion

In 2020, our institution embarked on a journey to high reliability with the goal of approaching zero harm. As part of efforts to promote a culture of safety, the hospital’s High Reliability Organization (HRO) team worked to develop a safety awards recognition program. Prior to the launch, the hospital’s patient safety committee recognized staff members through the medical center safety event reporting system (the Joint Patient Safety Reporting system [JPSR]) or through direct communication with staff members on safety actions they were engaged in. JPSR is the Veterans Health Administration National Center for Patient Safety incident reporting system for reporting, tracking, and trending of patient incidents in a national database. The award consisted of a certificate presented by the patient safety committee chairpersons to the employee in front of their peers in their respective work area. Hospital leadership was not involved in the safety awards recognition program at that time. No nomination process existed prior to our QI launch.

Once the QI initiative was launched and marketed heavily at staff meetings, we started to receive nominations for actions that were truly exceptional, while many others were submitted for behaviors that were within the day-to-day scope of practice of the staff member. For those early nominations that did not meet criteria for an award, we thanked staff for their submissions with a gentle statement that their nomination did not meet the criteria for an award. After following this practice for a few weeks, we became concerned that if we did not acknowledge the staff who came forward to request recognition for their routine work that supported safety, we could risk losing their engagement in a culture of safety. As such, we decided to create 3 levels of awards to recognize behaviors that went above and beyond while also acknowledging staff for actions within their scope of practice. Additionally, hospital leadership wanted to ensure that all staff recognize that their safety efforts are valued by leadership and that that sense of value will hopefully contribute to a culture of safety over time.

Initially, the single award system was called the “Good Catch Award” to acknowledge staff who go above and beyond to speak up and take action when they have safety concerns. This particular recognition includes a certificate, an encased baseball card that has been personalized by including the staff member’s picture and safety event identified, a stress-release baseball, and a stick of Bazooka gum (similar to what used to come in baseball cards packs). The award is presented to employees in their work area by the HRO and patient safety teams and includes representatives from the executive leadership team (ELT). The safety event identified is described by an ELT member, and all items are presented to the employee. Participation by the leadership team communicates how much the work being done to promote a culture of safety and advance quality health care is appreciated. This action also encourages others in the organization to identify and report safety concerns.¹³

With the rollout of the QI initiative, the volume of nominations submitted quickly increased (eg, approximately 1 every 2 months before to 3 per month following implementation). Frequently, nominations were for actions believed to be within the scope of the employee’s responsibilities. Our institution’s leadership team quickly recognized that, as an organization, not diminishing the importance of the “Good Catch Award” was important. However, the leadership team also wanted to encourage nominations from employees that involved safety issues that were part of the employee’s scope of responsibilities. As a result, 2 additional and equally notable award tiers were established, with specific criteria created for each.¹⁴ The addition of the other awards was instrumental in getting the leadership team to feel confident that all staff were being recognized for their commitment to patient safety.

The original Good Catch Award was labelled as a Level 1 award. The Level 2 safety recognition award, named the HRO Safety Champion Award, is given to employees who stop the line for a safety concern within their scope of practice and also participate as part of a team to investigate and improve processes to avoid recurring safety concerns in the future. For the Level Two award, a certificate is presented to an employee by the hospital’s HRO lead, HRO physician champion, patient safety manager, immediate supervisor, and peers. With the Level 3 award, the Culture of Safety Appreciation Award, individuals are recognized for addressing safety concerns within their assigned scope of responsibilities. Recognition is bestowed by an email of appreciation sent to the employee, acknowledging their commitment to promoting a culture of safety and quality health care. The recipient’s direct supervisor and other hospital leaders are copied on the message.¹⁴ See Table 1 for a comparison of awards.

Our institution’s HRO and patient safety teams utilized many additional venues to disseminate information regarding awardees and their actions. These included our monthly HRO newsletter, monthly safety forums, and biweekly Team Connecticut Healthcare system-wide huddles.

Nomination Process

Awards nominations are submitted via the hospital intranet homepage, where there is an “HRO Safety Award Nomination” icon. Once a staff member clicks the icon, a template opens asking for information, such as the reason for the nomination submission, and then walks them through the template using the CAR (C-context, A-actions, and R-results)¹⁵ format for describing the situation, identifying actions taken, and specifying the outcome of the action. Emails with award nominations can also be sent to the HRO lead, HRO champion, or Patient Safety Committee co-chairs. Calls for nominations are made at several venues attended by employees as well as supervisors. These include monthly safety forums, biweekly Team Connecticut Healthcare system-wide huddles, supervisory staff meetings, department and unit-based staff meetings, and many other formal and informal settings. This QI initiative has allowed us to capture potential awardees through several avenues, including self-nominations. All nominations are reviewed by a safety awards committee. Each committee member ranks the nomination as a Level 1, 2, or 3 award. For nominations where conflicting scores are obtained, the committee discusses the nomination together to resolve discrepancies.

Needed Resources

Material resources required for this QI initiative include certificate paper, plastic baseball card sleeves, stress-release baseballs, and Bazooka gum. The largest resource investment was the time needed to support the initiative. This included the time spent scheduling the Level 1 and 2 award presentations with staff and leadership. Time was also required to put the individual award packages together, which included printing the paper certificates, obtaining awardee pictures, placing them with their safety stories in a plastic baseball card sleeve, and arranging for the hospital photographer to take pictures of the awardees with their peers and leaders.

RESULTS

Prior to this QI initiative launch, 14 awards were given out over the preceding 2-year period. During the initial 18 months of the initiative (December 2020 to June 2022), 59 awards were presented (Level 1, n = 26; Level 2, n = 22; and Level 3, n = 11). Looking further into the Level 1 awards presented, 25 awardees worked in clinical roles and 1 in a nonclinical position (Table 2). The awardees represented multidisciplinary areas, including medical/surgical (med/surg) inpatient units, anesthesia, operating room, pharmacy, mental health clinics, surgical intensive care, specialty care clinics, and nutrition and food services. With the Level 2 awards, 18 clinical staff and 4 nonclinical staff received awards from the areas of med/surg inpatient, outpatient surgical suites, the medical center director’s office, radiology, pharmacy, primary care, facilities management, environmental management, infection prevention, and emergency services. All Level 3 awardees were from clinical areas, including primary care, hospital education, sterile processing, pharmacies, operating rooms, and med/surg inpatient units.

With the inception of this QI initiative, our organization has begun to see trends reflecting increased reporting of both actual and close-call events in JPSR (Figure 1).

With the inclusion of information regarding awardees and their actions in monthly safety forums, attendance at these forums has increased from an average of 64 attendees per month in 2021 to an average of 131 attendees per month in 2022 (Figure 2).

Finally, our organization’s annual All-Employee Survey results have shown incremental increases in staff reporting feeling psychologically safe and not fearing reprisal (Figure 3). It is important to note that there may be other contributing factors to these incremental changes.

Stories From the 3 Award Categories

Level 1 – Good Catch Award. M.S. was assigned as a continuous safety monitor, or “sitter,” on one of the med/surg inpatient units. M.S. arrived at the bedside and asked for a report on the patient at a change in shift. The report stated that the patient was sleeping and had not moved in a while. M.S. set about to perform the functions of a sitter and did her usual tasks in cleaning and tidying the room for the patient for breakfast and taking care of all items in the room, in general. M.S. introduced herself to the patient, who she thought might wake up because of her speaking to him. She thought the patient was in an odd position, and knowing that a patient should be a little further up in the bed, she tried with touch to awaken him to adjust his position. M.S. found that the patient was rather chilly to the touch and immediately became concerned. She continued to attempt to rouse the patient. M.S. called for the nurse and began to adjust the patient’s position. M.S. insisted that the patient was cold and “something was wrong.” A set of vitals was taken and a rapid response team code was called. The patient was immediately transferred to the intensive care unit to receive a higher level of care. If not for the diligence and caring attitude of M.S., this patient may have had a very poor outcome.

Reason for criteria being met: The scope of practice of a sitter is to be present in a patient’s room to monitor for falls and overall safety. This employee noticed that the patient was not responsive to verbal or tactile stimuli. Her immediate reporting of her concern to the nurse resulted in prompt intervention. If she had let the patient be, the patient could have died. The staff member went above and beyond by speaking up and taking action when she had a patient safety concern.

Level 2 – HRO Safety Champion Award. A patient presented to an outpatient clinic for monoclonal antibody (mAb) therapy for a COVID-19 infection; the treatment has been scheduled by the patient’s primary care provider. At that time, outpatient mAb therapy was the recommended care option for patients stable enough to receive treatment in this setting, but it is contraindicated in patients who are too unstable to receive mAb therapy in an outpatient setting, such as those with increased oxygen demands. R.L., a staff nurse, assessed the patient on arrival and found that his vital signs were stable, except for a slightly elevated respiratory rate. Upon questioning, the patient reported that he had increased his oxygen use at home from 2 to 4 L via a nasal cannula. R.L. assessed that the patient was too high-risk for outpatient mAb therapy and had the patient checked into the emergency department (ED) to receive a full diagnostic workup and evaluation by Dr. W., an ED provider. The patient required admission to the hospital for a higher level of care in an inpatient unit because of severe COVID-19 infection. Within 48 hours of admission, the patient’s condition further declined, requiring an upgrade to the medical intensive care unit with progressive interventions. Owing to the clinical assessment skills and prompt action of R.L., the patient was admitted to the hospital instead of receiving treatment in a suboptimal care setting and returning home. Had the patient gone home, his rapid decline could have had serious consequences.

Reason for criteria being met: On a cursory look, the patient may have passed as someone sufficiently stable to undergo outpatient treatment. However, the nurse stopped the line, paid close attention, and picked up on an abnormal vital sign and the projected consequences. The nurse brought the patient to a higher level of care in the ED so that he could get the attention he needed. If this patient was given mAb therapy in the outpatient setting, he would have been discharged and become sicker with the COVID-19 illness. As a result of this incident, R.L. is working with the outpatient clinic and ED staff to enahance triage and evaluation of patients referred for outpatient therapy for COVID-19 infections to prevent a similar event from recurring.

Level 3 – Culture of Safety Appreciation Award. While C.C. was reviewing the hazardous item competencies of the acute psychiatric inpatient staff, it was learned that staff were sniffing patients’ personal items to see if they were “safe” and free from alcohol. This is a potentially dangerous practice, and if fentanyl is present, it can be life-threatening. All patients admitted to acute inpatient psychiatry have all their clothing and personal items checked for hazardous items—pockets are emptied, soles of shoes are lifted, and so on. Staff wear personal protective equipment during this process to mitigate any powders or other harmful substances being inhaled or coming in contact with their skin or clothes. The gloves can be punctured if needles are found in the patient’s belongings. C.C. not only educated the staff on the dangers of sniffing for alcohol during hazardous-item checks, but also looked for further potential safety concerns. An additional identified risk was for needle sticks when such items were found in a patient’s belongings. C.C.’s recommendations included best practices to allow only unopened personal items and have available hospital-issued products as needed. C.C. remembered having a conversation with an employee from the psychiatric emergency room regarding the purchase of puncture-proof gloves to mitigate puncture sticks. C.C. recommended that the same gloves be used by staff on the acute inpatient psychiatry unit during searches for hazardous items.

Reason for criteria being met: The employee works in the hospital education department. It is within her scope of responsibilities to provide ongoing education to staff in order to address potential safety concerns.

DISCUSSION

This QI initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety and advancing quality health care, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. As part of efforts to continuously build on a safety-first culture, transparency and celebration of successes were demonstrated. This QI initiative demonstrated that a diverse and wide range of employees were reached, from clinical to nonclinical staff, and frontline to supervisory staff, as all were included in the recognition process. While many award nominations were received through the submission of safety concerns to the high-reliability team and patient safety office, several came directly from staff who wanted to recognize their peers for their work, supporting a culture of safety. This showed that staff felt that taking the time to submit a write-up to recognize a peer was an important task. Achieving zero harm for patients and staff alike is a top priority for our institution and guides all decisions, which reinforces that everyone has a responsibility to ensure that safety is always the first consideration. A culture of safety is enhanced by staff recognition. This QI initiative also showed that staff felt valued when they were acknowledged, regardless of the level of recognition they received. The theme of feeling valued came from unsolicited feedback. For example, some direct comments from awardees are presented in the Box.

In addition to endorsing the importance of safe practices to staff, safety award programs can identify gaps in existing standard procedures that can be updated quickly and shared broadly across a health care organization. The authors observed that the existence of the award program gives staff permission to use their voice to speak up when they have questions or concerns related to safety and to proactively engage in safety practices; a cultural shift of this kind informs safety practices and procedures and contributes to a more inspiring workplace. Staff at our organization who have received any of the safety awards, and those who are aware of these awards, have embraced the program readily. At the time of submission of this manuscript, there was a relative paucity of published literature on the details, performance, and impact of such programs. This initiative aims to share a road map highlighting the various dimensions of staff recognition and how the program supports our health care system in fostering a strong, sustainable culture of safety and health care quality. A next step is to formally assess the impact of the awards program on our culture of safety and quality using a psychometrically sound measurement tool, as recommended by TJC,¹⁶ such as the Hospital Survey on Patient Safety Culture.^17,18

CONCLUSION

A health care organization safety awards program is a strategy for building and sustaining a culture of safety. This QI initiative may be valuable to other organizations in the process of establishing a safety awards program of their own. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Corresponding author: John S. Murray, PhD, MPH, MSGH, RN, FAAN, 20 Chapel Street, Unit A502, Brookline, MA 02446; JMurray325@aol.com

Disclosures: None reported.

ABSTRACT

Objective: Promoting a culture of safety is a critical component of improving health care quality. Recognizing staff who stop the line for safety can positively impact the growth of a culture of safety. The purpose of this initiative was to demonstrate to staff the importance of speaking up for safety and being acknowledged for doing so.

Methods: Following a review of the literature on safety awards programs and their role in promoting a culture of safety in health care covering the period 2017 to 2020, a formal process was developed and implemented to disseminate safety awards to employees.

Results: During the initial 18 months of the initiative, a total of 59 awards were presented. The awards were well received by the recipients and other staff members. Within this period, adjustments were made to enhance the scope and reach of the program.

Conclusion: Recognizing staff behaviors that support a culture of safety is important for improving health care quality and employee engagement. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Keywords: patient safety, culture of safety, incident reporting, near miss.

A key aspect of improving health care quality is promoting and sustaining a culture of safety in the workplace. Improving the quality of health care services and systems involves making informed choices regarding the types of strategies to implement.¹ An essential aspect of supporting a safety culture is safety-event reporting. To approach the goal of zero harm, all safety events, whether they result in actual harm or are considered near misses, need to be reported. Near-miss events are errors that occur while care is being provided but are detected and corrected before harm reaches the patient.^1-3 Near-miss reporting plays a critical role in helping to identify and correct weaknesses in health care delivery systems and processes.⁴ However, evidence shows that there are a multitude of barriers to the reporting of near-miss events, such as fear of punitive actions, additional workload, unsupportive work environments, a culture with poor psychological safety, knowledge deficit, and lack of recognition of staff who do report near misses.^4-11

According to The Joint Commission (TJC), acknowledging health care team members who recognize and report unsafe conditions that provide insight for improving patient safety is a key method for promoting the reporting of near-miss events.⁶ As a result, some health care organizations and patient safety agencies have started to institute some form of recognition for their employees in the realm of safety.^8-10 The Pennsylvania Patient Safety Authority offers exceptional guidance for creating a safety awards program to promote a culture of safety.¹² Furthermore, TJC supports recognizing individuals and health care teams who identify and report near misses, or who have suggestions for initiatives to promote patient safety, with “good catch” awards. Individuals or teams working to promote and sustain a culture of safety should be recognized for their efforts. Acknowledging “good catches” to reward the identification, communication, and resolution of safety issues is an effective strategy for improving patient safety and health care quality.^6,8

This quality improvement (QI) initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. If health care organizations want staff to be motivated to report near misses and improve safety and health care quality, the culture needs to shift from focusing on blame to incentivizing individuals and teams to speak up when they have concerns.^8-10 Although deciding which safety actions are worthy of recognition can be challenging, recognizing all safe acts, regardless of how big or small they are perceived to be, is important. This QI initiative aimed to establish a tiered approach to recognize staff members for various categories of safety acts.

METHODS

A review of the literature from January 2017 to May 2020 for peer-reviewed publications regarding how other organizations implemented safety award programs to promote a culture of safety resulted in a dearth of evidence. This prompted us at the Veterans Affairs Connecticut Healthcare System to develop and implement a formal program to disseminate safety awards to employees.

Program Launch and Promotion

In 2020, our institution embarked on a journey to high reliability with the goal of approaching zero harm. As part of efforts to promote a culture of safety, the hospital’s High Reliability Organization (HRO) team worked to develop a safety awards recognition program. Prior to the launch, the hospital’s patient safety committee recognized staff members through the medical center safety event reporting system (the Joint Patient Safety Reporting system [JPSR]) or through direct communication with staff members on safety actions they were engaged in. JPSR is the Veterans Health Administration National Center for Patient Safety incident reporting system for reporting, tracking, and trending of patient incidents in a national database. The award consisted of a certificate presented by the patient safety committee chairpersons to the employee in front of their peers in their respective work area. Hospital leadership was not involved in the safety awards recognition program at that time. No nomination process existed prior to our QI launch.

Once the QI initiative was launched and marketed heavily at staff meetings, we started to receive nominations for actions that were truly exceptional, while many others were submitted for behaviors that were within the day-to-day scope of practice of the staff member. For those early nominations that did not meet criteria for an award, we thanked staff for their submissions with a gentle statement that their nomination did not meet the criteria for an award. After following this practice for a few weeks, we became concerned that if we did not acknowledge the staff who came forward to request recognition for their routine work that supported safety, we could risk losing their engagement in a culture of safety. As such, we decided to create 3 levels of awards to recognize behaviors that went above and beyond while also acknowledging staff for actions within their scope of practice. Additionally, hospital leadership wanted to ensure that all staff recognize that their safety efforts are valued by leadership and that that sense of value will hopefully contribute to a culture of safety over time.

Initially, the single award system was called the “Good Catch Award” to acknowledge staff who go above and beyond to speak up and take action when they have safety concerns. This particular recognition includes a certificate, an encased baseball card that has been personalized by including the staff member’s picture and safety event identified, a stress-release baseball, and a stick of Bazooka gum (similar to what used to come in baseball cards packs). The award is presented to employees in their work area by the HRO and patient safety teams and includes representatives from the executive leadership team (ELT). The safety event identified is described by an ELT member, and all items are presented to the employee. Participation by the leadership team communicates how much the work being done to promote a culture of safety and advance quality health care is appreciated. This action also encourages others in the organization to identify and report safety concerns.¹³

With the rollout of the QI initiative, the volume of nominations submitted quickly increased (eg, approximately 1 every 2 months before to 3 per month following implementation). Frequently, nominations were for actions believed to be within the scope of the employee’s responsibilities. Our institution’s leadership team quickly recognized that, as an organization, not diminishing the importance of the “Good Catch Award” was important. However, the leadership team also wanted to encourage nominations from employees that involved safety issues that were part of the employee’s scope of responsibilities. As a result, 2 additional and equally notable award tiers were established, with specific criteria created for each.¹⁴ The addition of the other awards was instrumental in getting the leadership team to feel confident that all staff were being recognized for their commitment to patient safety.

The original Good Catch Award was labelled as a Level 1 award. The Level 2 safety recognition award, named the HRO Safety Champion Award, is given to employees who stop the line for a safety concern within their scope of practice and also participate as part of a team to investigate and improve processes to avoid recurring safety concerns in the future. For the Level Two award, a certificate is presented to an employee by the hospital’s HRO lead, HRO physician champion, patient safety manager, immediate supervisor, and peers. With the Level 3 award, the Culture of Safety Appreciation Award, individuals are recognized for addressing safety concerns within their assigned scope of responsibilities. Recognition is bestowed by an email of appreciation sent to the employee, acknowledging their commitment to promoting a culture of safety and quality health care. The recipient’s direct supervisor and other hospital leaders are copied on the message.¹⁴ See Table 1 for a comparison of awards.

Our institution’s HRO and patient safety teams utilized many additional venues to disseminate information regarding awardees and their actions. These included our monthly HRO newsletter, monthly safety forums, and biweekly Team Connecticut Healthcare system-wide huddles.

Nomination Process

Awards nominations are submitted via the hospital intranet homepage, where there is an “HRO Safety Award Nomination” icon. Once a staff member clicks the icon, a template opens asking for information, such as the reason for the nomination submission, and then walks them through the template using the CAR (C-context, A-actions, and R-results)¹⁵ format for describing the situation, identifying actions taken, and specifying the outcome of the action. Emails with award nominations can also be sent to the HRO lead, HRO champion, or Patient Safety Committee co-chairs. Calls for nominations are made at several venues attended by employees as well as supervisors. These include monthly safety forums, biweekly Team Connecticut Healthcare system-wide huddles, supervisory staff meetings, department and unit-based staff meetings, and many other formal and informal settings. This QI initiative has allowed us to capture potential awardees through several avenues, including self-nominations. All nominations are reviewed by a safety awards committee. Each committee member ranks the nomination as a Level 1, 2, or 3 award. For nominations where conflicting scores are obtained, the committee discusses the nomination together to resolve discrepancies.

Needed Resources

Material resources required for this QI initiative include certificate paper, plastic baseball card sleeves, stress-release baseballs, and Bazooka gum. The largest resource investment was the time needed to support the initiative. This included the time spent scheduling the Level 1 and 2 award presentations with staff and leadership. Time was also required to put the individual award packages together, which included printing the paper certificates, obtaining awardee pictures, placing them with their safety stories in a plastic baseball card sleeve, and arranging for the hospital photographer to take pictures of the awardees with their peers and leaders.

RESULTS

Prior to this QI initiative launch, 14 awards were given out over the preceding 2-year period. During the initial 18 months of the initiative (December 2020 to June 2022), 59 awards were presented (Level 1, n = 26; Level 2, n = 22; and Level 3, n = 11). Looking further into the Level 1 awards presented, 25 awardees worked in clinical roles and 1 in a nonclinical position (Table 2). The awardees represented multidisciplinary areas, including medical/surgical (med/surg) inpatient units, anesthesia, operating room, pharmacy, mental health clinics, surgical intensive care, specialty care clinics, and nutrition and food services. With the Level 2 awards, 18 clinical staff and 4 nonclinical staff received awards from the areas of med/surg inpatient, outpatient surgical suites, the medical center director’s office, radiology, pharmacy, primary care, facilities management, environmental management, infection prevention, and emergency services. All Level 3 awardees were from clinical areas, including primary care, hospital education, sterile processing, pharmacies, operating rooms, and med/surg inpatient units.

With the inception of this QI initiative, our organization has begun to see trends reflecting increased reporting of both actual and close-call events in JPSR (Figure 1).

With the inclusion of information regarding awardees and their actions in monthly safety forums, attendance at these forums has increased from an average of 64 attendees per month in 2021 to an average of 131 attendees per month in 2022 (Figure 2).

Finally, our organization’s annual All-Employee Survey results have shown incremental increases in staff reporting feeling psychologically safe and not fearing reprisal (Figure 3). It is important to note that there may be other contributing factors to these incremental changes.

Stories From the 3 Award Categories

Level 1 – Good Catch Award. M.S. was assigned as a continuous safety monitor, or “sitter,” on one of the med/surg inpatient units. M.S. arrived at the bedside and asked for a report on the patient at a change in shift. The report stated that the patient was sleeping and had not moved in a while. M.S. set about to perform the functions of a sitter and did her usual tasks in cleaning and tidying the room for the patient for breakfast and taking care of all items in the room, in general. M.S. introduced herself to the patient, who she thought might wake up because of her speaking to him. She thought the patient was in an odd position, and knowing that a patient should be a little further up in the bed, she tried with touch to awaken him to adjust his position. M.S. found that the patient was rather chilly to the touch and immediately became concerned. She continued to attempt to rouse the patient. M.S. called for the nurse and began to adjust the patient’s position. M.S. insisted that the patient was cold and “something was wrong.” A set of vitals was taken and a rapid response team code was called. The patient was immediately transferred to the intensive care unit to receive a higher level of care. If not for the diligence and caring attitude of M.S., this patient may have had a very poor outcome.

Reason for criteria being met: The scope of practice of a sitter is to be present in a patient’s room to monitor for falls and overall safety. This employee noticed that the patient was not responsive to verbal or tactile stimuli. Her immediate reporting of her concern to the nurse resulted in prompt intervention. If she had let the patient be, the patient could have died. The staff member went above and beyond by speaking up and taking action when she had a patient safety concern.

Level 2 – HRO Safety Champion Award. A patient presented to an outpatient clinic for monoclonal antibody (mAb) therapy for a COVID-19 infection; the treatment has been scheduled by the patient’s primary care provider. At that time, outpatient mAb therapy was the recommended care option for patients stable enough to receive treatment in this setting, but it is contraindicated in patients who are too unstable to receive mAb therapy in an outpatient setting, such as those with increased oxygen demands. R.L., a staff nurse, assessed the patient on arrival and found that his vital signs were stable, except for a slightly elevated respiratory rate. Upon questioning, the patient reported that he had increased his oxygen use at home from 2 to 4 L via a nasal cannula. R.L. assessed that the patient was too high-risk for outpatient mAb therapy and had the patient checked into the emergency department (ED) to receive a full diagnostic workup and evaluation by Dr. W., an ED provider. The patient required admission to the hospital for a higher level of care in an inpatient unit because of severe COVID-19 infection. Within 48 hours of admission, the patient’s condition further declined, requiring an upgrade to the medical intensive care unit with progressive interventions. Owing to the clinical assessment skills and prompt action of R.L., the patient was admitted to the hospital instead of receiving treatment in a suboptimal care setting and returning home. Had the patient gone home, his rapid decline could have had serious consequences.

Reason for criteria being met: On a cursory look, the patient may have passed as someone sufficiently stable to undergo outpatient treatment. However, the nurse stopped the line, paid close attention, and picked up on an abnormal vital sign and the projected consequences. The nurse brought the patient to a higher level of care in the ED so that he could get the attention he needed. If this patient was given mAb therapy in the outpatient setting, he would have been discharged and become sicker with the COVID-19 illness. As a result of this incident, R.L. is working with the outpatient clinic and ED staff to enahance triage and evaluation of patients referred for outpatient therapy for COVID-19 infections to prevent a similar event from recurring.

Level 3 – Culture of Safety Appreciation Award. While C.C. was reviewing the hazardous item competencies of the acute psychiatric inpatient staff, it was learned that staff were sniffing patients’ personal items to see if they were “safe” and free from alcohol. This is a potentially dangerous practice, and if fentanyl is present, it can be life-threatening. All patients admitted to acute inpatient psychiatry have all their clothing and personal items checked for hazardous items—pockets are emptied, soles of shoes are lifted, and so on. Staff wear personal protective equipment during this process to mitigate any powders or other harmful substances being inhaled or coming in contact with their skin or clothes. The gloves can be punctured if needles are found in the patient’s belongings. C.C. not only educated the staff on the dangers of sniffing for alcohol during hazardous-item checks, but also looked for further potential safety concerns. An additional identified risk was for needle sticks when such items were found in a patient’s belongings. C.C.’s recommendations included best practices to allow only unopened personal items and have available hospital-issued products as needed. C.C. remembered having a conversation with an employee from the psychiatric emergency room regarding the purchase of puncture-proof gloves to mitigate puncture sticks. C.C. recommended that the same gloves be used by staff on the acute inpatient psychiatry unit during searches for hazardous items.

Reason for criteria being met: The employee works in the hospital education department. It is within her scope of responsibilities to provide ongoing education to staff in order to address potential safety concerns.

DISCUSSION

This QI initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety and advancing quality health care, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. As part of efforts to continuously build on a safety-first culture, transparency and celebration of successes were demonstrated. This QI initiative demonstrated that a diverse and wide range of employees were reached, from clinical to nonclinical staff, and frontline to supervisory staff, as all were included in the recognition process. While many award nominations were received through the submission of safety concerns to the high-reliability team and patient safety office, several came directly from staff who wanted to recognize their peers for their work, supporting a culture of safety. This showed that staff felt that taking the time to submit a write-up to recognize a peer was an important task. Achieving zero harm for patients and staff alike is a top priority for our institution and guides all decisions, which reinforces that everyone has a responsibility to ensure that safety is always the first consideration. A culture of safety is enhanced by staff recognition. This QI initiative also showed that staff felt valued when they were acknowledged, regardless of the level of recognition they received. The theme of feeling valued came from unsolicited feedback. For example, some direct comments from awardees are presented in the Box.

In addition to endorsing the importance of safe practices to staff, safety award programs can identify gaps in existing standard procedures that can be updated quickly and shared broadly across a health care organization. The authors observed that the existence of the award program gives staff permission to use their voice to speak up when they have questions or concerns related to safety and to proactively engage in safety practices; a cultural shift of this kind informs safety practices and procedures and contributes to a more inspiring workplace. Staff at our organization who have received any of the safety awards, and those who are aware of these awards, have embraced the program readily. At the time of submission of this manuscript, there was a relative paucity of published literature on the details, performance, and impact of such programs. This initiative aims to share a road map highlighting the various dimensions of staff recognition and how the program supports our health care system in fostering a strong, sustainable culture of safety and health care quality. A next step is to formally assess the impact of the awards program on our culture of safety and quality using a psychometrically sound measurement tool, as recommended by TJC,¹⁶ such as the Hospital Survey on Patient Safety Culture.^17,18

CONCLUSION

A health care organization safety awards program is a strategy for building and sustaining a culture of safety. This QI initiative may be valuable to other organizations in the process of establishing a safety awards program of their own. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Corresponding author: John S. Murray, PhD, MPH, MSGH, RN, FAAN, 20 Chapel Street, Unit A502, Brookline, MA 02446; JMurray325@aol.com

Disclosures: None reported.

ABSTRACT

Objective: Promoting a culture of safety is a critical component of improving health care quality. Recognizing staff who stop the line for safety can positively impact the growth of a culture of safety. The purpose of this initiative was to demonstrate to staff the importance of speaking up for safety and being acknowledged for doing so.

Methods: Following a review of the literature on safety awards programs and their role in promoting a culture of safety in health care covering the period 2017 to 2020, a formal process was developed and implemented to disseminate safety awards to employees.

Results: During the initial 18 months of the initiative, a total of 59 awards were presented. The awards were well received by the recipients and other staff members. Within this period, adjustments were made to enhance the scope and reach of the program.

Conclusion: Recognizing staff behaviors that support a culture of safety is important for improving health care quality and employee engagement. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Keywords: patient safety, culture of safety, incident reporting, near miss.

A key aspect of improving health care quality is promoting and sustaining a culture of safety in the workplace. Improving the quality of health care services and systems involves making informed choices regarding the types of strategies to implement.¹ An essential aspect of supporting a safety culture is safety-event reporting. To approach the goal of zero harm, all safety events, whether they result in actual harm or are considered near misses, need to be reported. Near-miss events are errors that occur while care is being provided but are detected and corrected before harm reaches the patient.^1-3 Near-miss reporting plays a critical role in helping to identify and correct weaknesses in health care delivery systems and processes.⁴ However, evidence shows that there are a multitude of barriers to the reporting of near-miss events, such as fear of punitive actions, additional workload, unsupportive work environments, a culture with poor psychological safety, knowledge deficit, and lack of recognition of staff who do report near misses.^4-11

According to The Joint Commission (TJC), acknowledging health care team members who recognize and report unsafe conditions that provide insight for improving patient safety is a key method for promoting the reporting of near-miss events.⁶ As a result, some health care organizations and patient safety agencies have started to institute some form of recognition for their employees in the realm of safety.^8-10 The Pennsylvania Patient Safety Authority offers exceptional guidance for creating a safety awards program to promote a culture of safety.¹² Furthermore, TJC supports recognizing individuals and health care teams who identify and report near misses, or who have suggestions for initiatives to promote patient safety, with “good catch” awards. Individuals or teams working to promote and sustain a culture of safety should be recognized for their efforts. Acknowledging “good catches” to reward the identification, communication, and resolution of safety issues is an effective strategy for improving patient safety and health care quality.^6,8

This quality improvement (QI) initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. If health care organizations want staff to be motivated to report near misses and improve safety and health care quality, the culture needs to shift from focusing on blame to incentivizing individuals and teams to speak up when they have concerns.^8-10 Although deciding which safety actions are worthy of recognition can be challenging, recognizing all safe acts, regardless of how big or small they are perceived to be, is important. This QI initiative aimed to establish a tiered approach to recognize staff members for various categories of safety acts.

METHODS

A review of the literature from January 2017 to May 2020 for peer-reviewed publications regarding how other organizations implemented safety award programs to promote a culture of safety resulted in a dearth of evidence. This prompted us at the Veterans Affairs Connecticut Healthcare System to develop and implement a formal program to disseminate safety awards to employees.

Program Launch and Promotion

In 2020, our institution embarked on a journey to high reliability with the goal of approaching zero harm. As part of efforts to promote a culture of safety, the hospital’s High Reliability Organization (HRO) team worked to develop a safety awards recognition program. Prior to the launch, the hospital’s patient safety committee recognized staff members through the medical center safety event reporting system (the Joint Patient Safety Reporting system [JPSR]) or through direct communication with staff members on safety actions they were engaged in. JPSR is the Veterans Health Administration National Center for Patient Safety incident reporting system for reporting, tracking, and trending of patient incidents in a national database. The award consisted of a certificate presented by the patient safety committee chairpersons to the employee in front of their peers in their respective work area. Hospital leadership was not involved in the safety awards recognition program at that time. No nomination process existed prior to our QI launch.

Once the QI initiative was launched and marketed heavily at staff meetings, we started to receive nominations for actions that were truly exceptional, while many others were submitted for behaviors that were within the day-to-day scope of practice of the staff member. For those early nominations that did not meet criteria for an award, we thanked staff for their submissions with a gentle statement that their nomination did not meet the criteria for an award. After following this practice for a few weeks, we became concerned that if we did not acknowledge the staff who came forward to request recognition for their routine work that supported safety, we could risk losing their engagement in a culture of safety. As such, we decided to create 3 levels of awards to recognize behaviors that went above and beyond while also acknowledging staff for actions within their scope of practice. Additionally, hospital leadership wanted to ensure that all staff recognize that their safety efforts are valued by leadership and that that sense of value will hopefully contribute to a culture of safety over time.

Initially, the single award system was called the “Good Catch Award” to acknowledge staff who go above and beyond to speak up and take action when they have safety concerns. This particular recognition includes a certificate, an encased baseball card that has been personalized by including the staff member’s picture and safety event identified, a stress-release baseball, and a stick of Bazooka gum (similar to what used to come in baseball cards packs). The award is presented to employees in their work area by the HRO and patient safety teams and includes representatives from the executive leadership team (ELT). The safety event identified is described by an ELT member, and all items are presented to the employee. Participation by the leadership team communicates how much the work being done to promote a culture of safety and advance quality health care is appreciated. This action also encourages others in the organization to identify and report safety concerns.¹³

With the rollout of the QI initiative, the volume of nominations submitted quickly increased (eg, approximately 1 every 2 months before to 3 per month following implementation). Frequently, nominations were for actions believed to be within the scope of the employee’s responsibilities. Our institution’s leadership team quickly recognized that, as an organization, not diminishing the importance of the “Good Catch Award” was important. However, the leadership team also wanted to encourage nominations from employees that involved safety issues that were part of the employee’s scope of responsibilities. As a result, 2 additional and equally notable award tiers were established, with specific criteria created for each.¹⁴ The addition of the other awards was instrumental in getting the leadership team to feel confident that all staff were being recognized for their commitment to patient safety.

The original Good Catch Award was labelled as a Level 1 award. The Level 2 safety recognition award, named the HRO Safety Champion Award, is given to employees who stop the line for a safety concern within their scope of practice and also participate as part of a team to investigate and improve processes to avoid recurring safety concerns in the future. For the Level Two award, a certificate is presented to an employee by the hospital’s HRO lead, HRO physician champion, patient safety manager, immediate supervisor, and peers. With the Level 3 award, the Culture of Safety Appreciation Award, individuals are recognized for addressing safety concerns within their assigned scope of responsibilities. Recognition is bestowed by an email of appreciation sent to the employee, acknowledging their commitment to promoting a culture of safety and quality health care. The recipient’s direct supervisor and other hospital leaders are copied on the message.¹⁴ See Table 1 for a comparison of awards.

Our institution’s HRO and patient safety teams utilized many additional venues to disseminate information regarding awardees and their actions. These included our monthly HRO newsletter, monthly safety forums, and biweekly Team Connecticut Healthcare system-wide huddles.

Nomination Process

Awards nominations are submitted via the hospital intranet homepage, where there is an “HRO Safety Award Nomination” icon. Once a staff member clicks the icon, a template opens asking for information, such as the reason for the nomination submission, and then walks them through the template using the CAR (C-context, A-actions, and R-results)¹⁵ format for describing the situation, identifying actions taken, and specifying the outcome of the action. Emails with award nominations can also be sent to the HRO lead, HRO champion, or Patient Safety Committee co-chairs. Calls for nominations are made at several venues attended by employees as well as supervisors. These include monthly safety forums, biweekly Team Connecticut Healthcare system-wide huddles, supervisory staff meetings, department and unit-based staff meetings, and many other formal and informal settings. This QI initiative has allowed us to capture potential awardees through several avenues, including self-nominations. All nominations are reviewed by a safety awards committee. Each committee member ranks the nomination as a Level 1, 2, or 3 award. For nominations where conflicting scores are obtained, the committee discusses the nomination together to resolve discrepancies.

Needed Resources

Material resources required for this QI initiative include certificate paper, plastic baseball card sleeves, stress-release baseballs, and Bazooka gum. The largest resource investment was the time needed to support the initiative. This included the time spent scheduling the Level 1 and 2 award presentations with staff and leadership. Time was also required to put the individual award packages together, which included printing the paper certificates, obtaining awardee pictures, placing them with their safety stories in a plastic baseball card sleeve, and arranging for the hospital photographer to take pictures of the awardees with their peers and leaders.

RESULTS

Prior to this QI initiative launch, 14 awards were given out over the preceding 2-year period. During the initial 18 months of the initiative (December 2020 to June 2022), 59 awards were presented (Level 1, n = 26; Level 2, n = 22; and Level 3, n = 11). Looking further into the Level 1 awards presented, 25 awardees worked in clinical roles and 1 in a nonclinical position (Table 2). The awardees represented multidisciplinary areas, including medical/surgical (med/surg) inpatient units, anesthesia, operating room, pharmacy, mental health clinics, surgical intensive care, specialty care clinics, and nutrition and food services. With the Level 2 awards, 18 clinical staff and 4 nonclinical staff received awards from the areas of med/surg inpatient, outpatient surgical suites, the medical center director’s office, radiology, pharmacy, primary care, facilities management, environmental management, infection prevention, and emergency services. All Level 3 awardees were from clinical areas, including primary care, hospital education, sterile processing, pharmacies, operating rooms, and med/surg inpatient units.

With the inception of this QI initiative, our organization has begun to see trends reflecting increased reporting of both actual and close-call events in JPSR (Figure 1).

With the inclusion of information regarding awardees and their actions in monthly safety forums, attendance at these forums has increased from an average of 64 attendees per month in 2021 to an average of 131 attendees per month in 2022 (Figure 2).

Finally, our organization’s annual All-Employee Survey results have shown incremental increases in staff reporting feeling psychologically safe and not fearing reprisal (Figure 3). It is important to note that there may be other contributing factors to these incremental changes.

Stories From the 3 Award Categories

Level 1 – Good Catch Award. M.S. was assigned as a continuous safety monitor, or “sitter,” on one of the med/surg inpatient units. M.S. arrived at the bedside and asked for a report on the patient at a change in shift. The report stated that the patient was sleeping and had not moved in a while. M.S. set about to perform the functions of a sitter and did her usual tasks in cleaning and tidying the room for the patient for breakfast and taking care of all items in the room, in general. M.S. introduced herself to the patient, who she thought might wake up because of her speaking to him. She thought the patient was in an odd position, and knowing that a patient should be a little further up in the bed, she tried with touch to awaken him to adjust his position. M.S. found that the patient was rather chilly to the touch and immediately became concerned. She continued to attempt to rouse the patient. M.S. called for the nurse and began to adjust the patient’s position. M.S. insisted that the patient was cold and “something was wrong.” A set of vitals was taken and a rapid response team code was called. The patient was immediately transferred to the intensive care unit to receive a higher level of care. If not for the diligence and caring attitude of M.S., this patient may have had a very poor outcome.

Reason for criteria being met: The scope of practice of a sitter is to be present in a patient’s room to monitor for falls and overall safety. This employee noticed that the patient was not responsive to verbal or tactile stimuli. Her immediate reporting of her concern to the nurse resulted in prompt intervention. If she had let the patient be, the patient could have died. The staff member went above and beyond by speaking up and taking action when she had a patient safety concern.

Level 2 – HRO Safety Champion Award. A patient presented to an outpatient clinic for monoclonal antibody (mAb) therapy for a COVID-19 infection; the treatment has been scheduled by the patient’s primary care provider. At that time, outpatient mAb therapy was the recommended care option for patients stable enough to receive treatment in this setting, but it is contraindicated in patients who are too unstable to receive mAb therapy in an outpatient setting, such as those with increased oxygen demands. R.L., a staff nurse, assessed the patient on arrival and found that his vital signs were stable, except for a slightly elevated respiratory rate. Upon questioning, the patient reported that he had increased his oxygen use at home from 2 to 4 L via a nasal cannula. R.L. assessed that the patient was too high-risk for outpatient mAb therapy and had the patient checked into the emergency department (ED) to receive a full diagnostic workup and evaluation by Dr. W., an ED provider. The patient required admission to the hospital for a higher level of care in an inpatient unit because of severe COVID-19 infection. Within 48 hours of admission, the patient’s condition further declined, requiring an upgrade to the medical intensive care unit with progressive interventions. Owing to the clinical assessment skills and prompt action of R.L., the patient was admitted to the hospital instead of receiving treatment in a suboptimal care setting and returning home. Had the patient gone home, his rapid decline could have had serious consequences.

Reason for criteria being met: On a cursory look, the patient may have passed as someone sufficiently stable to undergo outpatient treatment. However, the nurse stopped the line, paid close attention, and picked up on an abnormal vital sign and the projected consequences. The nurse brought the patient to a higher level of care in the ED so that he could get the attention he needed. If this patient was given mAb therapy in the outpatient setting, he would have been discharged and become sicker with the COVID-19 illness. As a result of this incident, R.L. is working with the outpatient clinic and ED staff to enahance triage and evaluation of patients referred for outpatient therapy for COVID-19 infections to prevent a similar event from recurring.

Level 3 – Culture of Safety Appreciation Award. While C.C. was reviewing the hazardous item competencies of the acute psychiatric inpatient staff, it was learned that staff were sniffing patients’ personal items to see if they were “safe” and free from alcohol. This is a potentially dangerous practice, and if fentanyl is present, it can be life-threatening. All patients admitted to acute inpatient psychiatry have all their clothing and personal items checked for hazardous items—pockets are emptied, soles of shoes are lifted, and so on. Staff wear personal protective equipment during this process to mitigate any powders or other harmful substances being inhaled or coming in contact with their skin or clothes. The gloves can be punctured if needles are found in the patient’s belongings. C.C. not only educated the staff on the dangers of sniffing for alcohol during hazardous-item checks, but also looked for further potential safety concerns. An additional identified risk was for needle sticks when such items were found in a patient’s belongings. C.C.’s recommendations included best practices to allow only unopened personal items and have available hospital-issued products as needed. C.C. remembered having a conversation with an employee from the psychiatric emergency room regarding the purchase of puncture-proof gloves to mitigate puncture sticks. C.C. recommended that the same gloves be used by staff on the acute inpatient psychiatry unit during searches for hazardous items.

Reason for criteria being met: The employee works in the hospital education department. It is within her scope of responsibilities to provide ongoing education to staff in order to address potential safety concerns.

DISCUSSION

This QI initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety and advancing quality health care, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. As part of efforts to continuously build on a safety-first culture, transparency and celebration of successes were demonstrated. This QI initiative demonstrated that a diverse and wide range of employees were reached, from clinical to nonclinical staff, and frontline to supervisory staff, as all were included in the recognition process. While many award nominations were received through the submission of safety concerns to the high-reliability team and patient safety office, several came directly from staff who wanted to recognize their peers for their work, supporting a culture of safety. This showed that staff felt that taking the time to submit a write-up to recognize a peer was an important task. Achieving zero harm for patients and staff alike is a top priority for our institution and guides all decisions, which reinforces that everyone has a responsibility to ensure that safety is always the first consideration. A culture of safety is enhanced by staff recognition. This QI initiative also showed that staff felt valued when they were acknowledged, regardless of the level of recognition they received. The theme of feeling valued came from unsolicited feedback. For example, some direct comments from awardees are presented in the Box.

In addition to endorsing the importance of safe practices to staff, safety award programs can identify gaps in existing standard procedures that can be updated quickly and shared broadly across a health care organization. The authors observed that the existence of the award program gives staff permission to use their voice to speak up when they have questions or concerns related to safety and to proactively engage in safety practices; a cultural shift of this kind informs safety practices and procedures and contributes to a more inspiring workplace. Staff at our organization who have received any of the safety awards, and those who are aware of these awards, have embraced the program readily. At the time of submission of this manuscript, there was a relative paucity of published literature on the details, performance, and impact of such programs. This initiative aims to share a road map highlighting the various dimensions of staff recognition and how the program supports our health care system in fostering a strong, sustainable culture of safety and health care quality. A next step is to formally assess the impact of the awards program on our culture of safety and quality using a psychometrically sound measurement tool, as recommended by TJC,¹⁶ such as the Hospital Survey on Patient Safety Culture.^17,18

CONCLUSION

A health care organization safety awards program is a strategy for building and sustaining a culture of safety. This QI initiative may be valuable to other organizations in the process of establishing a safety awards program of their own. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Corresponding author: John S. Murray, PhD, MPH, MSGH, RN, FAAN, 20 Chapel Street, Unit A502, Brookline, MA 02446; JMurray325@aol.com

Disclosures: None reported.

ABSTRACT

Objective: To teach internal medicine residents quality improvement (QI) principles in an effort to improve resident knowledge and comfort with QI, as well as address quality care gaps in resident clinic primary care patient panels.

Design: A QI curriculum was implemented for all residents rotating through a primary care block over a 6-month period. Residents completed Institute for Healthcare Improvement (IHI) modules, participated in a QI workshop, and received panel data reports, ultimately completing a plan-do-study-act (PDSA) cycle to improve colorectal cancer screening and hypertension control.

Setting and participants: This project was undertaken at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. All internal medicine residents were included, with 55 (73%) of the 75 residents completing the presurvey, and 39 (52%) completing the postsurvey.

Measurements: We administered a 10-question pre- and postsurvey looking at resident attitudes toward and comfort with QI and familiarity with their panel data as well as measured rates of colorectal cancer screening and hypertension control in resident panels.

Results: There was an increase in the numbers of residents who performed a PDSA cycle (P = .002), completed outreach based on their panel data (P = .02), and felt comfortable in both creating aim statements and designing and implementing PDSA cycles (P < .0001). The residents’ knowledge of their panel data significantly increased. There was no significant improvement in hypertension control, but there was an increase in colorectal cancer screening rates (P < .0001).

Conclusion: Providing panel data and performing targeted QI interventions can improve resident comfort with QI, translating to improvement in patient outcomes.

Keywords: quality improvement, resident education, medical education, care gaps, quality metrics.

As quality improvement (QI) has become an integral part of clinical practice, residency training programs have continued to evolve in how best to teach QI. The Accreditation Council for Graduate Medical Education (ACGME) Common Program requirements mandate that core competencies in residency programs include practice-based learning and improvement and systems-based practice.¹ Residents should receive education in QI, receive data on quality metrics and benchmarks related to their patient population, and participate in QI activities. The Clinical Learning Environment Review (CLER) program was established to provide feedback to institutions on 6 focused areas, including patient safety and health care quality. In visits to institutions across the United States, the CLER committees found that many residents had limited knowledge of QI concepts and limited access to data on quality metrics and benchmarks.²

There are many barriers to implementing a QI curriculum in residency programs, and creating and maintaining successful strategies has proven challenging.³ Many QI curricula for internal medicine residents have been described in the literature, but the results of many of these studies focus on resident self-assessment of QI knowledge and numbers of projects rather than on patient outcomes.^4-13 As there is some evidence suggesting that patients treated by residents have worse outcomes on ambulatory quality measures when compared with patients treated by staff physicians,^14,15 it is important to also look at patient outcomes when evaluating a QI curriculum. Experts in education recommend the following to optimize learning: exposure to both didactic and experiential opportunities, connection to health system improvement efforts, and assessment of patient outcomes in addition to learner feedback.^16,17 A study also found that providing panel data to residents could improve quality metrics.¹⁸

In this study, we sought to investigate the effects of a resident QI intervention during an ambulatory block on both residents’ self-assessments of QI knowledge and attitudes as well as on patient quality metrics.

Methods

Curriculum

We implemented this educational initiative at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. Co-located with the 415-bed academic medical center in downtown Boston, the practice serves more than 40,000 patients, approximately 7000 of whom are cared for by resident primary care physicians (PCPs). The internal medicine residents rotate through the primary care clinic as part of continuity clinic during ambulatory or elective blocks. In addition to continuity clinic, the residents have 2 dedicated 3-week primary care rotations during the course of an academic year. Primary care rotations consist of 5 clinic sessions a week as well as structured teaching sessions. Each resident inherits a panel of patients from an outgoing senior resident, with an average panel size of 96 patients per resident.

Prior to this study intervention, we did not do any formal QI teaching to our residents as part of their primary care curriculum, and previous panel management had focused more on chart reviews of patients whom residents perceived to be higher risk. Residents from all 3 years were included in the intervention. We taught a QI curriculum to our residents from January 2018 to June 2018 during the 3-week primary care rotation, which consisted of the following components:

• Institute for Healthcare Improvement (IHI) module QI 102 completed independently online.
• A 2-hour QI workshop led by 1 of 2 primary care faculty with backgrounds in QI, during which residents were taught basic principles of QI, including how to craft aim statements and design plan-do-study-act (PDSA) cycles, and participated in a hands-on QI activity designed to model rapid cycle improvement (the Paper Airplane Factory¹⁹).
• Distribution of individualized reports of residents’ patient panel data by email at the start of the primary care block that detailed patients’ overall rates of colorectal cancer screening and hypertension (HTN) control, along with the average resident panel rates and the average attending panel rates. The reports also included a list of all residents’ patients who were overdue for colorectal cancer screening or whose last blood pressure (BP) was uncontrolled (systolic BP ≥ 140 mm Hg or diastolic BP ≥  90 mm Hg). These reports were originally designed by our practice’s QI team and run and exported in Microsoft Excel format monthly by our information technology (IT) administrator.
• Instruction on aim statements as a group, followed by the expectation that each resident create an individualized aim statement tailored to each resident’s patient panel rates, with the PDSA cycle to be implemented during the remainder of the primary care rotation, focusing on improvement of colorectal cancer screening and HTN control (see supplementary eFigure 1 online for the worksheet used for the workshop).
• Residents were held accountable for their interventions by various check-ins. At the end of the primary care block, residents were required to submit their completed worksheets showing the intervention they had undertaken and when it was performed. The 2 primary care attendings primarily responsible for QI education would review the resident’s work approximately 1 to 2 months after they submitted their worksheets describing their intervention. These attendings sent the residents personalized feedback based on whether the intervention had been completed or successful as evidenced by documentation in the chart, including direct patient outreach by phone, letter, or portal; outreach to the resident coordinator; scheduled follow-up appointment; or booking or completion of colorectal cancer screening. Along with this feedback, residents were also sent suggestions for next steps. Resident preceptors were copied on the email to facilitate reinforcement of the goals and plans. Finally, the resident preceptors also helped with accountability by going through the residents’ worksheets and patient panel metrics with the residents during biannual evaluations.

Evaluation

Residents were surveyed with a 10-item questionnaire pre and post intervention regarding their attitudes toward QI, understanding of QI principles, and familiarity with their patient panel data. Surveys were anonymous and distributed via the SurveyMonkey platform (see supplementary eFigure 2 online). Residents were asked if they had ever performed a PDSA cycle, performed patient outreach, or performed an intervention and whether they knew the rates of diabetes, HTN, and colorectal cancer screening in their patient panels. Questions rated on a 5-point Likert scale were used to assess comfort with panel management, developing an aim statement, designing and implementing a PDSA cycle, as well as interest in pursuing QI as a career. For the purposes of analysis, these questions were dichotomized into “somewhat comfortable” and “very comfortable” vs “neutral,” “somewhat uncomfortable,” and “very uncomfortable.” Similarly, we dichotomized the question about interest in QI as a career into “somewhat interested” and “very interested” vs “neutral,” “somewhat disinterested,” and “very disinterested.” As the surveys were anonymous, we were unable to pair the pre- and postintervention surveys and used a chi-square test to evaluate whether there was an association between survey assessments pre intervention vs post intervention and a positive or negative response to the question.

We also examined rates of HTN control and colorectal cancer screening in all 75 resident panels pre and post intervention. The paired t-test was used to determine whether the mean change from pre to post intervention was significant. SAS 9.4 (SAS Institute Inc.) was used for all analyses. Institutional Review Board exemption was obtained from the Tufts Medical Center IRB. There was no funding received for this study.

Respondents

Of the 75 residents, 55 (73%) completed the survey prior to the intervention, and 39 (52%) completed the survey after the intervention.

Panel Knowledge and Intervention

Prior to the intervention, 45% of residents had performed a PDSA cycle, compared with 77% post intervention, which was a significant increase (P = .002) (Table 1). Sixty-two percent of residents had performed outreach or an intervention based on their patient panel reports prior to the intervention, compared with 85% of residents post intervention, which was also a significant increase (P = .02). The increase post intervention was not 100%, as there were residents who either missed the initial workshop or who did not follow through with their planned intervention. Common interventions included the residents giving their coordinators a list of patients to call to schedule appointments, utilizing fellow team members (eg, pharmacists, social workers) for targeted patient outreach, or calling patients themselves to reestablish a connection.

In terms of knowledge of their patient panels, prior to the intervention, 55%, 62%, and 62% of residents knew the rates of patients in their panel with diabetes, HTN, and colorectal cancer screening, respectively. After the intervention, the residents’ knowledge of these rates increased significantly, to 85% for diabetes (P = .002), 97% for HTN (P < .0001), and 97% for colorectal cancer screening (P < .0001).

Comfort With QI Approaches

Prior to the intervention, 82% of residents were comfortable managing their primary care panel, which did not change significantly post intervention (Table 2). The residents’ comfort with designing an aim statement did significantly increase, from 55% to 95% (P < .0001). The residents also had a significant increase in comfort with both designing and implementing a PDSA cycle. Prior to the intervention, 22% felt comfortable designing a PDSA cycle, which increased to 79% (P < .0001) post intervention, and 24% felt comfortable implementing a PDSA cycle, which increased to 77% (P < .0001) post intervention.

Patient Outcome Measures

The rate of HTN control in the residents' patient panels did not change significantly pre and post intervention (Table 3). The rate of resident patients who were up to date with colorectal cancer screening increased by 6.5% post intervention (P < .0001).

Interest in QI as a Career

As part of the survey, residents were asked how interested they were in making QI a part of their career. Fifty percent of residents indicated an interest in QI pre intervention, and 54% indicated an interest post intervention, which was not a significant difference (P = .72).

Discussion

In this study, we found that integration of a QI curriculum into a primary care rotation improved both residents’ knowledge of their patient panels and comfort with QI approaches, which translated to improvement in patient outcomes. Several previous studies have found improvements in resident self-assessment or knowledge after implementation of a QI curriculum.^4-13 Liao et al implemented a longitudinal curriculum including both didactic and experiential components and found an improvement in both QI confidence and knowledge.³ Similarly, Duello et al⁸ found that a curriculum including both didactic lectures and QI projects improved subjective QI knowledge and comfort. Interestingly, Fok and Wong⁹ found that resident knowledge could be sustained post curriculum after completion of a QI project, suggesting that experiential learning may be helpful in maintaining knowledge.

Studies also have looked at providing performance data to residents. Hwang et al¹⁸ found that providing audit and feedback in the form of individual panel performance data to residents compared with practice targets led to statistically significant improvement in cancer screening rates and composite quality score, indicating that there is tremendous potential in providing residents with their data. While the ACGME mandates that residents should receive data on their quality metrics, on CLER visits, many residents interviewed noted limited access to data on their metrics and benchmarks.^1,2

Though previous studies have individually looked at teaching QI concepts, providing panel data, or targeting select metrics, our study was unique in that it reviewed both self-reported resident outcomes data as well as actual patient outcomes. In addition to finding increased knowledge of patient panels and comfort with QI approaches, we found a significant increase in colorectal cancer screening rates post intervention. We thought this finding was particularly important given some data that residents' patients have been found to have worse outcomes on quality metrics compared with patients cared for by staff physicians.^14,15 Given that having a resident physician as a PCP has been associated with failing to meet quality measures, it is especially important to focus targeted quality improvement initiatives in this patient population to reduce disparities in care.

We found that residents had improved knowledge on their patient panels as a result of this initiative. The residents were noted to have a higher knowledge of their HTN and colorectal cancer screening rates in comparison to their diabetes metrics. We suspect this is because residents are provided with multiple metrics related to diabetes, including process measures such as A1c testing, as well as outcome measures such as A1c control, so it may be harder for them to elucidate exactly how they are doing with their diabetes patients, whereas in HTN control and colorectal cancer screening, there is only 1 associated metric. Interestingly, even though HTN and colorectal cancer screening were the 2 measures focused on in the study, the residents had a significant improvement in knowledge of the rates of diabetes in their panel as well. This suggests that even just receiving data alone is valuable, hopefully translating to better outcomes with better baseline understanding of panels. We believe that our intervention was successful because it included both a didactic and an experiential component, as well as the use of individual panel performance data.

There were several limitations to our study. It was performed at a single institution, translating to a small sample size. Our data analysis was limited because we were unable to pair our pre- and postintervention survey responses because we used an anonymous survey. We also did not have full participation in postintervention surveys from all residents, which may have biased the study in favor of high performers. Another limitation was that our survey relied on self-reported outcomes for the questions about the residents knowing their patient panels.

This study required a 2-hour workshop every 3 weeks led by a faculty member trained in QI. Given the amount of time needed for the curriculum, this study may be difficult to replicate at other institutions, especially if faculty with an interest or training in QI are not available. Given our finding that residents had increased knowledge of their patient panels after receiving panel metrics, simply providing data with the goal of smaller, focused interventions may be easier to implement. At our institution, we discontinued the longer 2-hour QI workshops designed to teach QI approaches more broadly. We continue to provide individualized panel data to all residents during their primary care rotations and conduct half-hour, small group workshops with the interns that focus on drafting aim statements and planning interventions. All residents are required to submit worksheets to us at the end of their primary care blocks listing their current rates of each predetermined metric and laying out their aim statements and planned interventions. Residents also continue to receive feedback from our faculty with expertise in QI afterward on their plans and evidence of follow-through in the chart, with their preceptors included on the feedback emails. Even without the larger QI workshop, this approach has continued to be successful and appreciated. In fact, it does appear as though improvement in colorectal cancer screening has been sustained over several years. At the end of our study period, the resident patient colorectal cancer screening rate rose from 34% to 43%, and for the 2021-2022 academic year, the rate rose further, from 46% to 50%.

Given that the resident clinic patient population is at higher risk overall, targeted outreach and approaches to improve quality must be continued. Future areas of research include looking at which interventions, whether QI curriculum, provision of panel data, or required panel management interventions, translate to the greatest improvements in patient outcomes in this vulnerable population.

Conclusion

Our study showed that a dedicated QI curriculum for the residents and access to quality metric data improved both resident knowledge and comfort with QI approaches. Beyond resident-centered outcomes, there was also translation to improved patient outcomes, with a significant increase in colon cancer screening rates post intervention.

Corresponding author: Kinjalika Sathi, MD, 800 Washington St., Boston, MA 02111; ksathi@tuftsmedicalcenter.org

Disclosures: None reported.

ABSTRACT

Objective: To teach internal medicine residents quality improvement (QI) principles in an effort to improve resident knowledge and comfort with QI, as well as address quality care gaps in resident clinic primary care patient panels.

Design: A QI curriculum was implemented for all residents rotating through a primary care block over a 6-month period. Residents completed Institute for Healthcare Improvement (IHI) modules, participated in a QI workshop, and received panel data reports, ultimately completing a plan-do-study-act (PDSA) cycle to improve colorectal cancer screening and hypertension control.

Setting and participants: This project was undertaken at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. All internal medicine residents were included, with 55 (73%) of the 75 residents completing the presurvey, and 39 (52%) completing the postsurvey.

Measurements: We administered a 10-question pre- and postsurvey looking at resident attitudes toward and comfort with QI and familiarity with their panel data as well as measured rates of colorectal cancer screening and hypertension control in resident panels.

Results: There was an increase in the numbers of residents who performed a PDSA cycle (P = .002), completed outreach based on their panel data (P = .02), and felt comfortable in both creating aim statements and designing and implementing PDSA cycles (P < .0001). The residents’ knowledge of their panel data significantly increased. There was no significant improvement in hypertension control, but there was an increase in colorectal cancer screening rates (P < .0001).

Conclusion: Providing panel data and performing targeted QI interventions can improve resident comfort with QI, translating to improvement in patient outcomes.

Keywords: quality improvement, resident education, medical education, care gaps, quality metrics.

As quality improvement (QI) has become an integral part of clinical practice, residency training programs have continued to evolve in how best to teach QI. The Accreditation Council for Graduate Medical Education (ACGME) Common Program requirements mandate that core competencies in residency programs include practice-based learning and improvement and systems-based practice.¹ Residents should receive education in QI, receive data on quality metrics and benchmarks related to their patient population, and participate in QI activities. The Clinical Learning Environment Review (CLER) program was established to provide feedback to institutions on 6 focused areas, including patient safety and health care quality. In visits to institutions across the United States, the CLER committees found that many residents had limited knowledge of QI concepts and limited access to data on quality metrics and benchmarks.²

There are many barriers to implementing a QI curriculum in residency programs, and creating and maintaining successful strategies has proven challenging.³ Many QI curricula for internal medicine residents have been described in the literature, but the results of many of these studies focus on resident self-assessment of QI knowledge and numbers of projects rather than on patient outcomes.^4-13 As there is some evidence suggesting that patients treated by residents have worse outcomes on ambulatory quality measures when compared with patients treated by staff physicians,^14,15 it is important to also look at patient outcomes when evaluating a QI curriculum. Experts in education recommend the following to optimize learning: exposure to both didactic and experiential opportunities, connection to health system improvement efforts, and assessment of patient outcomes in addition to learner feedback.^16,17 A study also found that providing panel data to residents could improve quality metrics.¹⁸

In this study, we sought to investigate the effects of a resident QI intervention during an ambulatory block on both residents’ self-assessments of QI knowledge and attitudes as well as on patient quality metrics.

Methods

Curriculum

We implemented this educational initiative at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. Co-located with the 415-bed academic medical center in downtown Boston, the practice serves more than 40,000 patients, approximately 7000 of whom are cared for by resident primary care physicians (PCPs). The internal medicine residents rotate through the primary care clinic as part of continuity clinic during ambulatory or elective blocks. In addition to continuity clinic, the residents have 2 dedicated 3-week primary care rotations during the course of an academic year. Primary care rotations consist of 5 clinic sessions a week as well as structured teaching sessions. Each resident inherits a panel of patients from an outgoing senior resident, with an average panel size of 96 patients per resident.

Prior to this study intervention, we did not do any formal QI teaching to our residents as part of their primary care curriculum, and previous panel management had focused more on chart reviews of patients whom residents perceived to be higher risk. Residents from all 3 years were included in the intervention. We taught a QI curriculum to our residents from January 2018 to June 2018 during the 3-week primary care rotation, which consisted of the following components:

• Institute for Healthcare Improvement (IHI) module QI 102 completed independently online.
• A 2-hour QI workshop led by 1 of 2 primary care faculty with backgrounds in QI, during which residents were taught basic principles of QI, including how to craft aim statements and design plan-do-study-act (PDSA) cycles, and participated in a hands-on QI activity designed to model rapid cycle improvement (the Paper Airplane Factory¹⁹).
• Distribution of individualized reports of residents’ patient panel data by email at the start of the primary care block that detailed patients’ overall rates of colorectal cancer screening and hypertension (HTN) control, along with the average resident panel rates and the average attending panel rates. The reports also included a list of all residents’ patients who were overdue for colorectal cancer screening or whose last blood pressure (BP) was uncontrolled (systolic BP ≥ 140 mm Hg or diastolic BP ≥  90 mm Hg). These reports were originally designed by our practice’s QI team and run and exported in Microsoft Excel format monthly by our information technology (IT) administrator.
• Instruction on aim statements as a group, followed by the expectation that each resident create an individualized aim statement tailored to each resident’s patient panel rates, with the PDSA cycle to be implemented during the remainder of the primary care rotation, focusing on improvement of colorectal cancer screening and HTN control (see supplementary eFigure 1 online for the worksheet used for the workshop).
• Residents were held accountable for their interventions by various check-ins. At the end of the primary care block, residents were required to submit their completed worksheets showing the intervention they had undertaken and when it was performed. The 2 primary care attendings primarily responsible for QI education would review the resident’s work approximately 1 to 2 months after they submitted their worksheets describing their intervention. These attendings sent the residents personalized feedback based on whether the intervention had been completed or successful as evidenced by documentation in the chart, including direct patient outreach by phone, letter, or portal; outreach to the resident coordinator; scheduled follow-up appointment; or booking or completion of colorectal cancer screening. Along with this feedback, residents were also sent suggestions for next steps. Resident preceptors were copied on the email to facilitate reinforcement of the goals and plans. Finally, the resident preceptors also helped with accountability by going through the residents’ worksheets and patient panel metrics with the residents during biannual evaluations.

Evaluation

Residents were surveyed with a 10-item questionnaire pre and post intervention regarding their attitudes toward QI, understanding of QI principles, and familiarity with their patient panel data. Surveys were anonymous and distributed via the SurveyMonkey platform (see supplementary eFigure 2 online). Residents were asked if they had ever performed a PDSA cycle, performed patient outreach, or performed an intervention and whether they knew the rates of diabetes, HTN, and colorectal cancer screening in their patient panels. Questions rated on a 5-point Likert scale were used to assess comfort with panel management, developing an aim statement, designing and implementing a PDSA cycle, as well as interest in pursuing QI as a career. For the purposes of analysis, these questions were dichotomized into “somewhat comfortable” and “very comfortable” vs “neutral,” “somewhat uncomfortable,” and “very uncomfortable.” Similarly, we dichotomized the question about interest in QI as a career into “somewhat interested” and “very interested” vs “neutral,” “somewhat disinterested,” and “very disinterested.” As the surveys were anonymous, we were unable to pair the pre- and postintervention surveys and used a chi-square test to evaluate whether there was an association between survey assessments pre intervention vs post intervention and a positive or negative response to the question.

We also examined rates of HTN control and colorectal cancer screening in all 75 resident panels pre and post intervention. The paired t-test was used to determine whether the mean change from pre to post intervention was significant. SAS 9.4 (SAS Institute Inc.) was used for all analyses. Institutional Review Board exemption was obtained from the Tufts Medical Center IRB. There was no funding received for this study.

Respondents

Of the 75 residents, 55 (73%) completed the survey prior to the intervention, and 39 (52%) completed the survey after the intervention.

Panel Knowledge and Intervention

Prior to the intervention, 45% of residents had performed a PDSA cycle, compared with 77% post intervention, which was a significant increase (P = .002) (Table 1). Sixty-two percent of residents had performed outreach or an intervention based on their patient panel reports prior to the intervention, compared with 85% of residents post intervention, which was also a significant increase (P = .02). The increase post intervention was not 100%, as there were residents who either missed the initial workshop or who did not follow through with their planned intervention. Common interventions included the residents giving their coordinators a list of patients to call to schedule appointments, utilizing fellow team members (eg, pharmacists, social workers) for targeted patient outreach, or calling patients themselves to reestablish a connection.

In terms of knowledge of their patient panels, prior to the intervention, 55%, 62%, and 62% of residents knew the rates of patients in their panel with diabetes, HTN, and colorectal cancer screening, respectively. After the intervention, the residents’ knowledge of these rates increased significantly, to 85% for diabetes (P = .002), 97% for HTN (P < .0001), and 97% for colorectal cancer screening (P < .0001).

Comfort With QI Approaches

Prior to the intervention, 82% of residents were comfortable managing their primary care panel, which did not change significantly post intervention (Table 2). The residents’ comfort with designing an aim statement did significantly increase, from 55% to 95% (P < .0001). The residents also had a significant increase in comfort with both designing and implementing a PDSA cycle. Prior to the intervention, 22% felt comfortable designing a PDSA cycle, which increased to 79% (P < .0001) post intervention, and 24% felt comfortable implementing a PDSA cycle, which increased to 77% (P < .0001) post intervention.

Patient Outcome Measures

The rate of HTN control in the residents' patient panels did not change significantly pre and post intervention (Table 3). The rate of resident patients who were up to date with colorectal cancer screening increased by 6.5% post intervention (P < .0001).

Interest in QI as a Career

As part of the survey, residents were asked how interested they were in making QI a part of their career. Fifty percent of residents indicated an interest in QI pre intervention, and 54% indicated an interest post intervention, which was not a significant difference (P = .72).

Discussion

In this study, we found that integration of a QI curriculum into a primary care rotation improved both residents’ knowledge of their patient panels and comfort with QI approaches, which translated to improvement in patient outcomes. Several previous studies have found improvements in resident self-assessment or knowledge after implementation of a QI curriculum.^4-13 Liao et al implemented a longitudinal curriculum including both didactic and experiential components and found an improvement in both QI confidence and knowledge.³ Similarly, Duello et al⁸ found that a curriculum including both didactic lectures and QI projects improved subjective QI knowledge and comfort. Interestingly, Fok and Wong⁹ found that resident knowledge could be sustained post curriculum after completion of a QI project, suggesting that experiential learning may be helpful in maintaining knowledge.

Studies also have looked at providing performance data to residents. Hwang et al¹⁸ found that providing audit and feedback in the form of individual panel performance data to residents compared with practice targets led to statistically significant improvement in cancer screening rates and composite quality score, indicating that there is tremendous potential in providing residents with their data. While the ACGME mandates that residents should receive data on their quality metrics, on CLER visits, many residents interviewed noted limited access to data on their metrics and benchmarks.^1,2

Though previous studies have individually looked at teaching QI concepts, providing panel data, or targeting select metrics, our study was unique in that it reviewed both self-reported resident outcomes data as well as actual patient outcomes. In addition to finding increased knowledge of patient panels and comfort with QI approaches, we found a significant increase in colorectal cancer screening rates post intervention. We thought this finding was particularly important given some data that residents' patients have been found to have worse outcomes on quality metrics compared with patients cared for by staff physicians.^14,15 Given that having a resident physician as a PCP has been associated with failing to meet quality measures, it is especially important to focus targeted quality improvement initiatives in this patient population to reduce disparities in care.

We found that residents had improved knowledge on their patient panels as a result of this initiative. The residents were noted to have a higher knowledge of their HTN and colorectal cancer screening rates in comparison to their diabetes metrics. We suspect this is because residents are provided with multiple metrics related to diabetes, including process measures such as A1c testing, as well as outcome measures such as A1c control, so it may be harder for them to elucidate exactly how they are doing with their diabetes patients, whereas in HTN control and colorectal cancer screening, there is only 1 associated metric. Interestingly, even though HTN and colorectal cancer screening were the 2 measures focused on in the study, the residents had a significant improvement in knowledge of the rates of diabetes in their panel as well. This suggests that even just receiving data alone is valuable, hopefully translating to better outcomes with better baseline understanding of panels. We believe that our intervention was successful because it included both a didactic and an experiential component, as well as the use of individual panel performance data.

There were several limitations to our study. It was performed at a single institution, translating to a small sample size. Our data analysis was limited because we were unable to pair our pre- and postintervention survey responses because we used an anonymous survey. We also did not have full participation in postintervention surveys from all residents, which may have biased the study in favor of high performers. Another limitation was that our survey relied on self-reported outcomes for the questions about the residents knowing their patient panels.

This study required a 2-hour workshop every 3 weeks led by a faculty member trained in QI. Given the amount of time needed for the curriculum, this study may be difficult to replicate at other institutions, especially if faculty with an interest or training in QI are not available. Given our finding that residents had increased knowledge of their patient panels after receiving panel metrics, simply providing data with the goal of smaller, focused interventions may be easier to implement. At our institution, we discontinued the longer 2-hour QI workshops designed to teach QI approaches more broadly. We continue to provide individualized panel data to all residents during their primary care rotations and conduct half-hour, small group workshops with the interns that focus on drafting aim statements and planning interventions. All residents are required to submit worksheets to us at the end of their primary care blocks listing their current rates of each predetermined metric and laying out their aim statements and planned interventions. Residents also continue to receive feedback from our faculty with expertise in QI afterward on their plans and evidence of follow-through in the chart, with their preceptors included on the feedback emails. Even without the larger QI workshop, this approach has continued to be successful and appreciated. In fact, it does appear as though improvement in colorectal cancer screening has been sustained over several years. At the end of our study period, the resident patient colorectal cancer screening rate rose from 34% to 43%, and for the 2021-2022 academic year, the rate rose further, from 46% to 50%.

Given that the resident clinic patient population is at higher risk overall, targeted outreach and approaches to improve quality must be continued. Future areas of research include looking at which interventions, whether QI curriculum, provision of panel data, or required panel management interventions, translate to the greatest improvements in patient outcomes in this vulnerable population.

Conclusion

Our study showed that a dedicated QI curriculum for the residents and access to quality metric data improved both resident knowledge and comfort with QI approaches. Beyond resident-centered outcomes, there was also translation to improved patient outcomes, with a significant increase in colon cancer screening rates post intervention.

Corresponding author: Kinjalika Sathi, MD, 800 Washington St., Boston, MA 02111; ksathi@tuftsmedicalcenter.org

Disclosures: None reported.

ABSTRACT

Objective: To teach internal medicine residents quality improvement (QI) principles in an effort to improve resident knowledge and comfort with QI, as well as address quality care gaps in resident clinic primary care patient panels.

Design: A QI curriculum was implemented for all residents rotating through a primary care block over a 6-month period. Residents completed Institute for Healthcare Improvement (IHI) modules, participated in a QI workshop, and received panel data reports, ultimately completing a plan-do-study-act (PDSA) cycle to improve colorectal cancer screening and hypertension control.

Setting and participants: This project was undertaken at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. All internal medicine residents were included, with 55 (73%) of the 75 residents completing the presurvey, and 39 (52%) completing the postsurvey.

Measurements: We administered a 10-question pre- and postsurvey looking at resident attitudes toward and comfort with QI and familiarity with their panel data as well as measured rates of colorectal cancer screening and hypertension control in resident panels.

Results: There was an increase in the numbers of residents who performed a PDSA cycle (P = .002), completed outreach based on their panel data (P = .02), and felt comfortable in both creating aim statements and designing and implementing PDSA cycles (P < .0001). The residents’ knowledge of their panel data significantly increased. There was no significant improvement in hypertension control, but there was an increase in colorectal cancer screening rates (P < .0001).

Conclusion: Providing panel data and performing targeted QI interventions can improve resident comfort with QI, translating to improvement in patient outcomes.

Keywords: quality improvement, resident education, medical education, care gaps, quality metrics.

As quality improvement (QI) has become an integral part of clinical practice, residency training programs have continued to evolve in how best to teach QI. The Accreditation Council for Graduate Medical Education (ACGME) Common Program requirements mandate that core competencies in residency programs include practice-based learning and improvement and systems-based practice.¹ Residents should receive education in QI, receive data on quality metrics and benchmarks related to their patient population, and participate in QI activities. The Clinical Learning Environment Review (CLER) program was established to provide feedback to institutions on 6 focused areas, including patient safety and health care quality. In visits to institutions across the United States, the CLER committees found that many residents had limited knowledge of QI concepts and limited access to data on quality metrics and benchmarks.²

There are many barriers to implementing a QI curriculum in residency programs, and creating and maintaining successful strategies has proven challenging.³ Many QI curricula for internal medicine residents have been described in the literature, but the results of many of these studies focus on resident self-assessment of QI knowledge and numbers of projects rather than on patient outcomes.^4-13 As there is some evidence suggesting that patients treated by residents have worse outcomes on ambulatory quality measures when compared with patients treated by staff physicians,^14,15 it is important to also look at patient outcomes when evaluating a QI curriculum. Experts in education recommend the following to optimize learning: exposure to both didactic and experiential opportunities, connection to health system improvement efforts, and assessment of patient outcomes in addition to learner feedback.^16,17 A study also found that providing panel data to residents could improve quality metrics.¹⁸

In this study, we sought to investigate the effects of a resident QI intervention during an ambulatory block on both residents’ self-assessments of QI knowledge and attitudes as well as on patient quality metrics.

Methods

Curriculum

We implemented this educational initiative at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. Co-located with the 415-bed academic medical center in downtown Boston, the practice serves more than 40,000 patients, approximately 7000 of whom are cared for by resident primary care physicians (PCPs). The internal medicine residents rotate through the primary care clinic as part of continuity clinic during ambulatory or elective blocks. In addition to continuity clinic, the residents have 2 dedicated 3-week primary care rotations during the course of an academic year. Primary care rotations consist of 5 clinic sessions a week as well as structured teaching sessions. Each resident inherits a panel of patients from an outgoing senior resident, with an average panel size of 96 patients per resident.

Prior to this study intervention, we did not do any formal QI teaching to our residents as part of their primary care curriculum, and previous panel management had focused more on chart reviews of patients whom residents perceived to be higher risk. Residents from all 3 years were included in the intervention. We taught a QI curriculum to our residents from January 2018 to June 2018 during the 3-week primary care rotation, which consisted of the following components:

• Institute for Healthcare Improvement (IHI) module QI 102 completed independently online.
• A 2-hour QI workshop led by 1 of 2 primary care faculty with backgrounds in QI, during which residents were taught basic principles of QI, including how to craft aim statements and design plan-do-study-act (PDSA) cycles, and participated in a hands-on QI activity designed to model rapid cycle improvement (the Paper Airplane Factory¹⁹).
• Distribution of individualized reports of residents’ patient panel data by email at the start of the primary care block that detailed patients’ overall rates of colorectal cancer screening and hypertension (HTN) control, along with the average resident panel rates and the average attending panel rates. The reports also included a list of all residents’ patients who were overdue for colorectal cancer screening or whose last blood pressure (BP) was uncontrolled (systolic BP ≥ 140 mm Hg or diastolic BP ≥  90 mm Hg). These reports were originally designed by our practice’s QI team and run and exported in Microsoft Excel format monthly by our information technology (IT) administrator.
• Instruction on aim statements as a group, followed by the expectation that each resident create an individualized aim statement tailored to each resident’s patient panel rates, with the PDSA cycle to be implemented during the remainder of the primary care rotation, focusing on improvement of colorectal cancer screening and HTN control (see supplementary eFigure 1 online for the worksheet used for the workshop).
• Residents were held accountable for their interventions by various check-ins. At the end of the primary care block, residents were required to submit their completed worksheets showing the intervention they had undertaken and when it was performed. The 2 primary care attendings primarily responsible for QI education would review the resident’s work approximately 1 to 2 months after they submitted their worksheets describing their intervention. These attendings sent the residents personalized feedback based on whether the intervention had been completed or successful as evidenced by documentation in the chart, including direct patient outreach by phone, letter, or portal; outreach to the resident coordinator; scheduled follow-up appointment; or booking or completion of colorectal cancer screening. Along with this feedback, residents were also sent suggestions for next steps. Resident preceptors were copied on the email to facilitate reinforcement of the goals and plans. Finally, the resident preceptors also helped with accountability by going through the residents’ worksheets and patient panel metrics with the residents during biannual evaluations.

Evaluation

Residents were surveyed with a 10-item questionnaire pre and post intervention regarding their attitudes toward QI, understanding of QI principles, and familiarity with their patient panel data. Surveys were anonymous and distributed via the SurveyMonkey platform (see supplementary eFigure 2 online). Residents were asked if they had ever performed a PDSA cycle, performed patient outreach, or performed an intervention and whether they knew the rates of diabetes, HTN, and colorectal cancer screening in their patient panels. Questions rated on a 5-point Likert scale were used to assess comfort with panel management, developing an aim statement, designing and implementing a PDSA cycle, as well as interest in pursuing QI as a career. For the purposes of analysis, these questions were dichotomized into “somewhat comfortable” and “very comfortable” vs “neutral,” “somewhat uncomfortable,” and “very uncomfortable.” Similarly, we dichotomized the question about interest in QI as a career into “somewhat interested” and “very interested” vs “neutral,” “somewhat disinterested,” and “very disinterested.” As the surveys were anonymous, we were unable to pair the pre- and postintervention surveys and used a chi-square test to evaluate whether there was an association between survey assessments pre intervention vs post intervention and a positive or negative response to the question.

We also examined rates of HTN control and colorectal cancer screening in all 75 resident panels pre and post intervention. The paired t-test was used to determine whether the mean change from pre to post intervention was significant. SAS 9.4 (SAS Institute Inc.) was used for all analyses. Institutional Review Board exemption was obtained from the Tufts Medical Center IRB. There was no funding received for this study.

Respondents

Of the 75 residents, 55 (73%) completed the survey prior to the intervention, and 39 (52%) completed the survey after the intervention.

Panel Knowledge and Intervention

Prior to the intervention, 45% of residents had performed a PDSA cycle, compared with 77% post intervention, which was a significant increase (P = .002) (Table 1). Sixty-two percent of residents had performed outreach or an intervention based on their patient panel reports prior to the intervention, compared with 85% of residents post intervention, which was also a significant increase (P = .02). The increase post intervention was not 100%, as there were residents who either missed the initial workshop or who did not follow through with their planned intervention. Common interventions included the residents giving their coordinators a list of patients to call to schedule appointments, utilizing fellow team members (eg, pharmacists, social workers) for targeted patient outreach, or calling patients themselves to reestablish a connection.

In terms of knowledge of their patient panels, prior to the intervention, 55%, 62%, and 62% of residents knew the rates of patients in their panel with diabetes, HTN, and colorectal cancer screening, respectively. After the intervention, the residents’ knowledge of these rates increased significantly, to 85% for diabetes (P = .002), 97% for HTN (P < .0001), and 97% for colorectal cancer screening (P < .0001).

Comfort With QI Approaches

Prior to the intervention, 82% of residents were comfortable managing their primary care panel, which did not change significantly post intervention (Table 2). The residents’ comfort with designing an aim statement did significantly increase, from 55% to 95% (P < .0001). The residents also had a significant increase in comfort with both designing and implementing a PDSA cycle. Prior to the intervention, 22% felt comfortable designing a PDSA cycle, which increased to 79% (P < .0001) post intervention, and 24% felt comfortable implementing a PDSA cycle, which increased to 77% (P < .0001) post intervention.

Patient Outcome Measures

The rate of HTN control in the residents' patient panels did not change significantly pre and post intervention (Table 3). The rate of resident patients who were up to date with colorectal cancer screening increased by 6.5% post intervention (P < .0001).

Interest in QI as a Career

As part of the survey, residents were asked how interested they were in making QI a part of their career. Fifty percent of residents indicated an interest in QI pre intervention, and 54% indicated an interest post intervention, which was not a significant difference (P = .72).

Discussion

In this study, we found that integration of a QI curriculum into a primary care rotation improved both residents’ knowledge of their patient panels and comfort with QI approaches, which translated to improvement in patient outcomes. Several previous studies have found improvements in resident self-assessment or knowledge after implementation of a QI curriculum.^4-13 Liao et al implemented a longitudinal curriculum including both didactic and experiential components and found an improvement in both QI confidence and knowledge.³ Similarly, Duello et al⁸ found that a curriculum including both didactic lectures and QI projects improved subjective QI knowledge and comfort. Interestingly, Fok and Wong⁹ found that resident knowledge could be sustained post curriculum after completion of a QI project, suggesting that experiential learning may be helpful in maintaining knowledge.

Studies also have looked at providing performance data to residents. Hwang et al¹⁸ found that providing audit and feedback in the form of individual panel performance data to residents compared with practice targets led to statistically significant improvement in cancer screening rates and composite quality score, indicating that there is tremendous potential in providing residents with their data. While the ACGME mandates that residents should receive data on their quality metrics, on CLER visits, many residents interviewed noted limited access to data on their metrics and benchmarks.^1,2

Though previous studies have individually looked at teaching QI concepts, providing panel data, or targeting select metrics, our study was unique in that it reviewed both self-reported resident outcomes data as well as actual patient outcomes. In addition to finding increased knowledge of patient panels and comfort with QI approaches, we found a significant increase in colorectal cancer screening rates post intervention. We thought this finding was particularly important given some data that residents' patients have been found to have worse outcomes on quality metrics compared with patients cared for by staff physicians.^14,15 Given that having a resident physician as a PCP has been associated with failing to meet quality measures, it is especially important to focus targeted quality improvement initiatives in this patient population to reduce disparities in care.

We found that residents had improved knowledge on their patient panels as a result of this initiative. The residents were noted to have a higher knowledge of their HTN and colorectal cancer screening rates in comparison to their diabetes metrics. We suspect this is because residents are provided with multiple metrics related to diabetes, including process measures such as A1c testing, as well as outcome measures such as A1c control, so it may be harder for them to elucidate exactly how they are doing with their diabetes patients, whereas in HTN control and colorectal cancer screening, there is only 1 associated metric. Interestingly, even though HTN and colorectal cancer screening were the 2 measures focused on in the study, the residents had a significant improvement in knowledge of the rates of diabetes in their panel as well. This suggests that even just receiving data alone is valuable, hopefully translating to better outcomes with better baseline understanding of panels. We believe that our intervention was successful because it included both a didactic and an experiential component, as well as the use of individual panel performance data.

There were several limitations to our study. It was performed at a single institution, translating to a small sample size. Our data analysis was limited because we were unable to pair our pre- and postintervention survey responses because we used an anonymous survey. We also did not have full participation in postintervention surveys from all residents, which may have biased the study in favor of high performers. Another limitation was that our survey relied on self-reported outcomes for the questions about the residents knowing their patient panels.

This study required a 2-hour workshop every 3 weeks led by a faculty member trained in QI. Given the amount of time needed for the curriculum, this study may be difficult to replicate at other institutions, especially if faculty with an interest or training in QI are not available. Given our finding that residents had increased knowledge of their patient panels after receiving panel metrics, simply providing data with the goal of smaller, focused interventions may be easier to implement. At our institution, we discontinued the longer 2-hour QI workshops designed to teach QI approaches more broadly. We continue to provide individualized panel data to all residents during their primary care rotations and conduct half-hour, small group workshops with the interns that focus on drafting aim statements and planning interventions. All residents are required to submit worksheets to us at the end of their primary care blocks listing their current rates of each predetermined metric and laying out their aim statements and planned interventions. Residents also continue to receive feedback from our faculty with expertise in QI afterward on their plans and evidence of follow-through in the chart, with their preceptors included on the feedback emails. Even without the larger QI workshop, this approach has continued to be successful and appreciated. In fact, it does appear as though improvement in colorectal cancer screening has been sustained over several years. At the end of our study period, the resident patient colorectal cancer screening rate rose from 34% to 43%, and for the 2021-2022 academic year, the rate rose further, from 46% to 50%.

Given that the resident clinic patient population is at higher risk overall, targeted outreach and approaches to improve quality must be continued. Future areas of research include looking at which interventions, whether QI curriculum, provision of panel data, or required panel management interventions, translate to the greatest improvements in patient outcomes in this vulnerable population.

Conclusion

Our study showed that a dedicated QI curriculum for the residents and access to quality metric data improved both resident knowledge and comfort with QI approaches. Beyond resident-centered outcomes, there was also translation to improved patient outcomes, with a significant increase in colon cancer screening rates post intervention.

Corresponding author: Kinjalika Sathi, MD, 800 Washington St., Boston, MA 02111; ksathi@tuftsmedicalcenter.org

Disclosures: None reported.

Abstract

Diagnostic errors in hospitalized patients are a leading cause of preventable morbidity and mortality. Significant challenges in defining and measuring diagnostic errors and underlying process failure points have led to considerable variability in reported rates of diagnostic errors and adverse outcomes. In this article, we explore the diagnostic process and its discrete components, emphasizing the centrality of the patient in decision-making as well as the continuous nature of the process. We review the incidence of diagnostic errors in hospitalized patients and different methodological approaches that have been used to arrive at these estimates. We discuss different but interdependent provider- and system-related process-failure points that lead to diagnostic errors. We examine specific challenges related to measurement of diagnostic errors and describe traditional and novel approaches that are being used to obtain the most precise estimates. Finally, we examine various patient-, provider-, and organizational-level interventions that have been proposed to improve diagnostic safety in hospitalized patients.

Keywords: diagnostic error, hospital medicine, patient safety.

Diagnosis is defined as a “pre-existing set of categories agreed upon by the medical profession to designate a specific condition.”¹ The diagnostic process involves obtaining a clinical history, performing a physical examination, conducting diagnostic testing, and consulting with other clinical providers to gather data that are relevant to understanding the underlying disease processes. This exercise involves generating hypotheses and updating prior probabilities as more information and evidence become available. Throughout this process of information gathering, integration, and interpretation, there is an ongoing assessment of whether sufficient and necessary knowledge has been obtained to make an accurate diagnosis and provide appropriate treatment.²

Diagnostic error is defined as a missed opportunity to make a timely diagnosis as part of this iterative process, including the failure of communicating the diagnosis to the patient in a timely manner.³ It can be categorized as a missed, delayed, or incorrect diagnosis based on available evidence at the time. Establishing the correct diagnosis has important implications. A timely and precise diagnosis ensures the patient the highest probability of having a positive health outcome that reflects an appropriate understanding of underlying disease processes and is consistent with their overall goals of care.³ When diagnostic errors occur, they can cause patient harm. Adverse events due to medical errors, including diagnostic errors, are estimated to be the third leading cause of death in the United States.⁴ Most people will experience at least 1 diagnostic error in their lifetime. In the 2015 National Academy of Medicine report Improving Diagnosis in Healthcare, diagnostic errors were identified as a major hazard as well as an opportunity to improve patient outcomes.²

Diagnostic errors during hospitalizations are especially concerning, as they are more likely to be implicated in a wider spectrum of harm, including permanent disability and death. This has become even more relevant for hospital medicine physicians and other clinical providers as they encounter increasing cognitive and administrative workloads, rising dissatisfaction and burnout, and unique obstacles such as night-time scheduling.⁵

Incidence of Diagnostic Errors in Hospitalized Patients

Several methodological approaches have been used to estimate the incidence of diagnostic errors in hospitalized patients. These include retrospective reviews of a sample of all hospital admissions, evaluations of selected adverse outcomes including autopsy studies, patient and provider surveys, and malpractice claims. Laboratory testing audits and secondary reviews in other diagnostic subspecialities (eg, radiology, pathology, and microbiology) are also essential to improving diagnostic performance in these specialized fields, which in turn affects overall hospital diagnostic error rates.^6-8 These diverse approaches provide unique insights regarding our ability to assess the degree to which potential harms, ranging from temporary impairment to permanent disability, to death, are attributable to different failure points in the diagnostic process.

Large retrospective chart reviews of random hospital admissions remain the most accurate way to determine the overall incidence of diagnostic errors in hospitalized patients.⁹ The Harvard Medical Practice Study, published in 1991, laid the groundwork for measuring the incidence of adverse events in hospitalized patients and assessing their relation to medical error, negligence, and disability. Reviewing 30,121 randomly selected records from 51 randomly selected acute care hospitals in New York State, the study found that adverse events occurred in 3.7% of hospitalizations, diagnostic errors accounted for 13.8% of these events, and these errors were likely attributable to negligence in 74.7% of cases. The study not only outlined individual-level process failures, but also focused attention on some of the systemic causes, setting the agenda for quality improvement research in hospital-based care for years to come.^10-12 A recent systematic review and meta-analysis of 22 hospital admission studies found a pooled rate of 0.7% (95% CI, 0.5%-1.1%) for harmful diagnostic errors.⁹ It found significant variations in the rates of adverse events, diagnostic errors, and range of diagnoses that were missed. This was primarily because of variabilities in pre-test probabilities in detecting diagnostic errors in these specific cohorts, as well as due to heterogeneity in study definitions and methodologies, especially regarding how they defined and measured “diagnostic error.” The analysis, however, did not account for diagnostic errors that were not related to patient harm (missed opportunities); therefore, it likely significantly underestimated the true incidence of diagnostic errors in these study populations. Table 1 summarizes some of key studies that have examined the incidence of harmful diagnostic errors in hospitalized patients.^9-21

The chief limitation of reviewing random hospital admissions is that, since overall rates of diagnostic errors are still relatively low, a large number of case reviews are required to identify a sufficient sample of adverse outcomes to gain a meaningful understanding of the underlying process failure points and develop tools for remediation. Patient and provider surveys or data from malpractice claims can be high-yield starting points for research on process errors.^22,23 Reviews of enriched cohorts of adverse outcomes, such as rapid-response events, intensive care unit (ICU) transfers, deaths, and hospital readmissions, can be an efficient way to identify process failures that lead to greatest harm. Depending on the research approach and the types of underlying patient populations sampled, rates of diagnostic errors in these high-risk groups have been estimated to be approximately 5% to 20%, or even higher.^6,24-31 For example, a retrospective study of 391 cases of unplanned 7-day readmissions found that 5.6% of cases contained at least 1 diagnostic error during the index admission.³² In a study conducted at 6 Belgian acute-care hospitals, 56% of patients requiring an unplanned transfer to a higher level of care were determined to have had an adverse event, and of these adverse events, 12.4% of cases were associated with errors in diagnosis.²⁹ A systematic review of 16 hospital-based studies estimated that 3.1% of all inpatient deaths were likely preventable, which corresponded to 22,165 deaths annually in the United States.³⁰ Another such review of 31 autopsy studies reported that 28% of autopsied ICU patients had at least 1 misdiagnosis; of these diagnostic errors, 8% were classified as potentially lethal, and 15% were considered major but not lethal.³¹ Significant drawbacks of such enriched cohort studies, however, are their poor generalizability and inability to detect failure points that do not lead to patient harm (near-miss events).³³

Causes of Diagnostic Errors in Hospitalized Patients

All aspects of the diagnostic process are susceptible to errors. These errors stem from a variety of faulty processes, including failure of the patient to engage with the health care system (eg, due to lack of insurance or transportation, or delay in seeking care); failure in information gathering (eg, missed history or exam findings, ordering wrong tests, laboratory errors); failure in information interpretation (eg, exam finding or test result misinterpretation); inaccurate hypothesis generation (eg, due to suboptimal prioritization or weighing of supporting evidence); and failure in communication (eg, with other team members or with the patient).^2,34 Reasons for diagnostic process failures vary widely across different health care settings. While clinician assessment errors (eg, failure to consider or alternatively overweigh competing diagnoses) and errors in testing and the monitoring phase (eg, failure to order or follow up diagnostic tests) can lead to a majority of diagnostic errors in some patient populations, in other settings, social (eg, poor health literacy, punitive cultural practices) and economic factors (eg, lack of access to appropriate diagnostic tests or to specialty expertise) play a more prominent role.^34,35

The Figure describes the relationship between components of the diagnostic process and subsequent outcomes, including diagnostic process failures, diagnostic errors, and absence or presence of patient harm.^2,36,37 It reemphasizes the centrality of the patient in decision-making and the continuous nature of the process. The Figure also illustrates that only a minority of process failures result in diagnostic errors, and a smaller proportion of diagnostic errors actually lead to patient harm. Conversely, it also shows that diagnostic errors can happen without any obvious process-failure points, and, similarly, patient harm can take place in the absence of any evident diagnostic errors.^36-38 Finally, it highlights the need to incorporate feedback from process failures, diagnostic errors, and favorable and unfavorable patient outcomes in order to inform future quality improvement efforts and research.

A significant proportion of diagnostic errors are due to system-related vulnerabilities, such as limitations in availability, adoption or quality of work force training, health informatics resources, and diagnostic capabilities. Lack of institutional culture that promotes safety and transparency also predisposes to diagnostic errors.^39,40 The other major domain of process failures is related to cognitive errors in clinician decision-making. Anchoring, confirmation bias, availability bias, and base-rate neglect are some of the common cognitive biases that, along with personality traits (aversion to risk or ambiguity, overconfidence) and affective biases (influence of emotion on decision-making), often determine the degree of utilization of resources and the possibility of suboptimal diagnostic performance.^41,42 Further, implicit biases related to age, race, gender, and sexual orientation contribute to disparities in access to health care and outcomes.⁴³ In a large number of cases of preventable adverse outcomes, however, there are multiple interdependent individual and system-related failure points that lead to diagnostic error and patient harm.^6,32

Challenges in Defining and Measuring Diagnostic Errors

In order to develop effective, evidence-based interventions to reduce diagnostic errors in hospitalized patients, it is essential to be able to first operationally define, and then accurately measure, diagnostic errors and the process failures that contribute to these errors in a standardized way that is reproducible across different settings.^6,44 There are a number of obstacles in this endeavor.

A fundamental problem is that establishing a diagnosis is not a single act but a process. Patterns of symptoms and clinical presentations often differ for the same disease. Information required to make a diagnosis is usually gathered in stages, where the clinician obtains additional data, while considering many possibilities, of which 1 may be ultimately correct. Diagnoses evolve over time and in different care settings. “The most likely diagnosis” is not always the same as “the final correct diagnosis.” Moreover, the diagnostic process is influenced by patients’ individual clinical courses and preferences over time. This makes determination of missed, delayed, or incorrect diagnoses challenging.^45,46

For hospitalized patients, generally the goal is to first rule out more serious and acute conditions (eg, pulmonary embolism or stroke), even if their probability is rather low. Conversely, a diagnosis that appears less consequential if delayed (eg, chronic anemia of unclear etiology) might not be pursued on an urgent basis, and is often left to outpatient providers to examine, but still may manifest in downstream harm (eg, delayed diagnosis of gastrointestinal malignancy or recurrent admissions for heart failure due to missed iron-deficiency anemia). Therefore, coming up with disease diagnosis likelihoods in hindsight may turn out to be highly subjective and not always accurate. This can be particularly difficult when clinician and other team deliberations are not recorded in their entirety.⁴⁷

Another hurdle in the practice of diagnostic medicine is to preserve the balance between underdiagnosing versus pursuing overly aggressive diagnostic approaches. Conducting laboratory, imaging, or other diagnostic studies without a clear shared understanding of how they would affect clinical decision-making (eg, use of prostate-specific antigen to detect prostate cancer) not only leads to increased costs but can also delay appropriate care. Worse, subsequent unnecessary diagnostic tests and treatments can sometimes lead to serious harm.^48,49

Finally, retrospective reviews by clinicians are subject to multiple potential limitations that include failure to create well-defined research questions, poorly developed inclusion and exclusion criteria, and issues related to inter- and intra-rater reliability.⁵⁰ These methodological deficiencies can occur despite following "best practice" guidelines during the study planning, execution, and analysis phases. They further add to the challenge of defining and measuring diagnostic errors.⁴⁷

Strategies to Improve Measurement of Diagnostic Errors

Development of new methodologies to reliably measure diagnostic errors is an area of active research. The advancement of uniform and universally agreed-upon frameworks to define and identify process failure points and diagnostic errors would help reduce measurement error and support development and testing of interventions that could be generalizable across different health care settings. To more accurately define and measure diagnostic errors, several novel approaches have been proposed (Table 2).

The Safer Dx framework is an all-round tool developed to advance the discipline of measuring diagnostic errors. For an episode of care under review, the instrument scores various items to determine the likelihood of a diagnostic error. These items evaluate multiple dimensions affecting diagnostic performance and measurements across 3 broad domains: structure (provider and organizational characteristics—from everyone involved with patient care, to computing infrastructure, to policies and regulations), process (elements of the patient-provider encounter, diagnostic test performance and follow-up, and subspecialty- and referral-specific factors), and outcome (establishing accurate and timely diagnosis as opposed to missed, delayed, or incorrect diagnosis). This instrument has been revised and can be further modified by a variety of stakeholders, including clinicians, health care organizations, and policymakers, to identify potential diagnostic errors in a standardized way for patient safety and quality improvement research.^51,52

Use of standardized tools, such as the Diagnosis Error Evaluation and Research (DEER) taxonomy, can help to identify and classify specific failure points across different diagnostic process dimensions.³⁷ These failure points can be classified into: issues related to patient presentation or access to health care; failure to obtain or misinterpretation of history or physical exam findings; errors in use of diagnostics tests due to technical or clinician-related factors; failures in appropriate weighing of evidence and hypothesis generation; errors associated with referral or consultation process; and failure to monitor the patient or obtain timely follow-up.³⁴ The DEER taxonomy can also be modified based on specific research questions and study populations. Further, it can be recategorized to correspond to Safer Dx framework diagnostic process dimensions to provide insights into reasons for specific process failures and to develop new interventions to mitigate errors and patient harm.⁶

Since a majority of diagnostic errors do not lead to actual harm, use of “triggers” or clues (eg, procedure-related complications, patient falls, transfers to a higher level of care, readmissions within 30 days) can be a more efficient method to identify diagnostic errors and adverse events that do cause harm. The Global Trigger Tool, developed by the Institute for Healthcare Improvement, uses this strategy. This tool has been shown to identify a significantly higher number of serious adverse events than comparable methods.⁵³ This facilitates selection and development of strategies at the institutional level that are most likely to improve patient outcomes.²⁴

Encouraging and facilitating voluntary or prompted reporting from patients and clinicians can also play an important role in capturing diagnostic errors. Patients and clinicians are not only the key stakeholders but are also uniquely placed within the diagnostic process to detect and report potential errors.^25,54 Patient-safety-event reporting systems, such as RL6, play a vital role in reporting near-misses and adverse events. These systems provide a mechanism for team members at all levels within the hospital to contribute toward reporting patient adverse events, including those arising from diagnostic errors.⁵⁵ The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is the first standardized, nationally reported patient survey designed to measure patients’ perceptions of their hospital experience. The US Centers for Medicare and Medicaid Services (CMS) publishes HCAHPS results on its website 4 times a year, which serves as an important incentive for hospitals to improve patient safety and quality of health care delivery.⁵⁶

Another novel approach links multiple symptoms to a range of target diseases using the Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) framework. Using “big data” technologies, this technique can help discover otherwise hidden symptom-disease links and improve overall diagnostic performance. This approach is proposed for both case-control (look-back) and cohort (look-forward) studies assessing diagnostic errors and misdiagnosis-related harms. For example, starting with a known diagnosis with high potential for harm (eg, stroke), the “look-back” approach can be used to identify high-risk symptoms (eg, dizziness, vertigo). In the “look-forward” approach, a single symptom or exposure risk factor known to be frequently misdiagnosed (eg, dizziness) can be analyzed to identify potential adverse disease outcomes (eg, stroke, migraine).⁵⁷

Many large ongoing studies looking at diagnostic errors among hospitalized patients, such as Utility of Predictive Systems to identify Inpatient Diagnostic Errors (UPSIDE),⁵⁸Patient Safety Learning Lab (PSLL),⁵⁹ and Achieving Diagnostic Excellence through Prevention and Teamwork (ADEPT),⁶⁰ are using structured chart review methodologies incorporating many of the above strategies in combination. Cases triggered by certain events (eg, ICU transfer, death, rapid response event, new or worsening acute kidney injury) are reviewed using validated tools, including Safer Dx framework and DEER taxonomy, to provide the most precise estimates of the burden of diagnostic errors in hospitalized patients. These estimates may be much higher than previously predicted using traditional chart review approaches.^6,24 For example, a recently published study of 2809 random admissions in 11 Massachusetts hospitals identified 978 adverse events but only 10 diagnostic errors (diagnostic error rate, 0.4%).¹⁹ This was likely because the trigger method used in the study did not specifically examine the diagnostic process as critically as done by the Safer Dx framework and DEER taxonomy tools, thereby underestimating the total number of diagnostic errors. Further, these ongoing studies (eg, UPSIDE, ADEPT) aim to employ new and upcoming advanced machine-learning methods to create models that can improve overall diagnostic performance. This would pave the way to test and build novel, efficient, and scalable interventions to reduce diagnostic errors and improve patient outcomes.

Strategies to Improve Diagnostic Safety in Hospitalized Patients

Disease-specific biomedical research, as well as advances in laboratory, imaging, and other technologies, play a critical role in improving diagnostic accuracy. However, these technical approaches do not address many of the broader clinician- and system-level failure points and opportunities for improvement. Various patient-, provider-, and organizational-level interventions that could make diagnostic processes more resilient and reduce the risk of error and patient harm have been proposed.⁶¹

Among these strategies are approaches to empower patients and their families. Fostering therapeutic relationships between patients and members of the care team is essential to reducing diagnostic errors.⁶² Facilitating timely access to health records, ensuring transparency in decision making, and tailoring communication strategies to patients’ cultural and educational backgrounds can reduce harm.⁶³ Similarly, at the system level, enhancing communication among different providers by use of tools such as structured handoffs can prevent communication breakdowns and facilitate positive outcomes.⁶⁴

Interventions targeted at individual health care providers, such as educational programs to improve content-specific knowledge, can enhance diagnostic performance. Regular feedback, strategies to enhance equity, and fostering an environment where all providers are actively encouraged to think critically and participate in the diagnostic process (training programs to use “diagnostic time-outs” and making it a “team sport”) can improve clinical reasoning.^65,66 Use of standardized patients can help identify individual-level cognitive failure points and facilitate creation of new interventions to improve clinical decision-making processes.⁶⁷

Novel health information technologies can further augment these efforts. These include effective documentation by maintaining dynamic and accurate patient histories, problem lists, and medication lists^68-70; use of electronic health record–based algorithms to identify potential diagnostic delays for serious conditions^71,72; use of telemedicine technologies to improve accessibility and coordination⁷³;application of mobile health and wearable technologies to facilitate data-gathering and care delivery^74,75; and use of computerized decision-support tools, including applications to interpret electrocardiograms, imaging studies, and other diagnostic tests.⁷⁶

Use of precision medicine, powered by new artificial intelligence (AI) tools, is becoming more widespread. Algorithms powered by AI can augment and sometimes even outperform clinician decision-making in areas such as oncology, radiology, and primary care.⁷⁷ Creation of large biobanks like the All of Us research program can be used to study thousands of environmental and genetic risk factors and health conditions simultaneously, and help identify specific treatments that work best for people of different backgrounds.⁷⁸ Active research in these areas holds great promise in terms of how and when we diagnose diseases and make appropriate preventative and treatment decisions. Significant scientific, ethical, and regulatory challenges will need to be overcome before these technologies can address some of the most complex problems in health care.⁷⁹

Finally, diagnostic performance is affected by the external environment, including the functioning of the medical liability system. Diagnostic errors that lead to patient harm are a leading cause of malpractice claims.⁸⁰ Developing a legal environment, in collaboration with patient advocacy groups and health care organizations, that promotes and facilitates timely disclosure of diagnostic errors could decrease the incentive to hide errors, advance care processes, and improve outcomes.^81,82

Conclusion

The burden of diagnostic errors in hospitalized patients is unacceptably high and remains an underemphasized cause of preventable morbidity and mortality. Diagnostic errors often result from a breakdown in multiple interdependent processes that involve patient-, provider-, and system-level factors. Significant challenges remain in defining and identifying diagnostic errors as well as underlying process-failure points. The most effective interventions to reduce diagnostic errors will require greater patient participation in the diagnostic process and a mix of evidence-based interventions that promote individual-provider excellence as well as system-level changes. Further research and collaboration among various stakeholders should help improve diagnostic safety for hospitalized patients.

Corresponding author: Abhishek Goyal, MD, MPH; agoyal4@bwh.harvard.edu

Disclosures: Dr. Dalal disclosed receiving income ≥ $250 from MayaMD.

Abstract

Diagnostic errors in hospitalized patients are a leading cause of preventable morbidity and mortality. Significant challenges in defining and measuring diagnostic errors and underlying process failure points have led to considerable variability in reported rates of diagnostic errors and adverse outcomes. In this article, we explore the diagnostic process and its discrete components, emphasizing the centrality of the patient in decision-making as well as the continuous nature of the process. We review the incidence of diagnostic errors in hospitalized patients and different methodological approaches that have been used to arrive at these estimates. We discuss different but interdependent provider- and system-related process-failure points that lead to diagnostic errors. We examine specific challenges related to measurement of diagnostic errors and describe traditional and novel approaches that are being used to obtain the most precise estimates. Finally, we examine various patient-, provider-, and organizational-level interventions that have been proposed to improve diagnostic safety in hospitalized patients.

Keywords: diagnostic error, hospital medicine, patient safety.

Diagnosis is defined as a “pre-existing set of categories agreed upon by the medical profession to designate a specific condition.”¹ The diagnostic process involves obtaining a clinical history, performing a physical examination, conducting diagnostic testing, and consulting with other clinical providers to gather data that are relevant to understanding the underlying disease processes. This exercise involves generating hypotheses and updating prior probabilities as more information and evidence become available. Throughout this process of information gathering, integration, and interpretation, there is an ongoing assessment of whether sufficient and necessary knowledge has been obtained to make an accurate diagnosis and provide appropriate treatment.²

Diagnostic error is defined as a missed opportunity to make a timely diagnosis as part of this iterative process, including the failure of communicating the diagnosis to the patient in a timely manner.³ It can be categorized as a missed, delayed, or incorrect diagnosis based on available evidence at the time. Establishing the correct diagnosis has important implications. A timely and precise diagnosis ensures the patient the highest probability of having a positive health outcome that reflects an appropriate understanding of underlying disease processes and is consistent with their overall goals of care.³ When diagnostic errors occur, they can cause patient harm. Adverse events due to medical errors, including diagnostic errors, are estimated to be the third leading cause of death in the United States.⁴ Most people will experience at least 1 diagnostic error in their lifetime. In the 2015 National Academy of Medicine report Improving Diagnosis in Healthcare, diagnostic errors were identified as a major hazard as well as an opportunity to improve patient outcomes.²

Diagnostic errors during hospitalizations are especially concerning, as they are more likely to be implicated in a wider spectrum of harm, including permanent disability and death. This has become even more relevant for hospital medicine physicians and other clinical providers as they encounter increasing cognitive and administrative workloads, rising dissatisfaction and burnout, and unique obstacles such as night-time scheduling.⁵

Incidence of Diagnostic Errors in Hospitalized Patients

Several methodological approaches have been used to estimate the incidence of diagnostic errors in hospitalized patients. These include retrospective reviews of a sample of all hospital admissions, evaluations of selected adverse outcomes including autopsy studies, patient and provider surveys, and malpractice claims. Laboratory testing audits and secondary reviews in other diagnostic subspecialities (eg, radiology, pathology, and microbiology) are also essential to improving diagnostic performance in these specialized fields, which in turn affects overall hospital diagnostic error rates.^6-8 These diverse approaches provide unique insights regarding our ability to assess the degree to which potential harms, ranging from temporary impairment to permanent disability, to death, are attributable to different failure points in the diagnostic process.

Large retrospective chart reviews of random hospital admissions remain the most accurate way to determine the overall incidence of diagnostic errors in hospitalized patients.⁹ The Harvard Medical Practice Study, published in 1991, laid the groundwork for measuring the incidence of adverse events in hospitalized patients and assessing their relation to medical error, negligence, and disability. Reviewing 30,121 randomly selected records from 51 randomly selected acute care hospitals in New York State, the study found that adverse events occurred in 3.7% of hospitalizations, diagnostic errors accounted for 13.8% of these events, and these errors were likely attributable to negligence in 74.7% of cases. The study not only outlined individual-level process failures, but also focused attention on some of the systemic causes, setting the agenda for quality improvement research in hospital-based care for years to come.^10-12 A recent systematic review and meta-analysis of 22 hospital admission studies found a pooled rate of 0.7% (95% CI, 0.5%-1.1%) for harmful diagnostic errors.⁹ It found significant variations in the rates of adverse events, diagnostic errors, and range of diagnoses that were missed. This was primarily because of variabilities in pre-test probabilities in detecting diagnostic errors in these specific cohorts, as well as due to heterogeneity in study definitions and methodologies, especially regarding how they defined and measured “diagnostic error.” The analysis, however, did not account for diagnostic errors that were not related to patient harm (missed opportunities); therefore, it likely significantly underestimated the true incidence of diagnostic errors in these study populations. Table 1 summarizes some of key studies that have examined the incidence of harmful diagnostic errors in hospitalized patients.^9-21

The chief limitation of reviewing random hospital admissions is that, since overall rates of diagnostic errors are still relatively low, a large number of case reviews are required to identify a sufficient sample of adverse outcomes to gain a meaningful understanding of the underlying process failure points and develop tools for remediation. Patient and provider surveys or data from malpractice claims can be high-yield starting points for research on process errors.^22,23 Reviews of enriched cohorts of adverse outcomes, such as rapid-response events, intensive care unit (ICU) transfers, deaths, and hospital readmissions, can be an efficient way to identify process failures that lead to greatest harm. Depending on the research approach and the types of underlying patient populations sampled, rates of diagnostic errors in these high-risk groups have been estimated to be approximately 5% to 20%, or even higher.^6,24-31 For example, a retrospective study of 391 cases of unplanned 7-day readmissions found that 5.6% of cases contained at least 1 diagnostic error during the index admission.³² In a study conducted at 6 Belgian acute-care hospitals, 56% of patients requiring an unplanned transfer to a higher level of care were determined to have had an adverse event, and of these adverse events, 12.4% of cases were associated with errors in diagnosis.²⁹ A systematic review of 16 hospital-based studies estimated that 3.1% of all inpatient deaths were likely preventable, which corresponded to 22,165 deaths annually in the United States.³⁰ Another such review of 31 autopsy studies reported that 28% of autopsied ICU patients had at least 1 misdiagnosis; of these diagnostic errors, 8% were classified as potentially lethal, and 15% were considered major but not lethal.³¹ Significant drawbacks of such enriched cohort studies, however, are their poor generalizability and inability to detect failure points that do not lead to patient harm (near-miss events).³³

Causes of Diagnostic Errors in Hospitalized Patients

All aspects of the diagnostic process are susceptible to errors. These errors stem from a variety of faulty processes, including failure of the patient to engage with the health care system (eg, due to lack of insurance or transportation, or delay in seeking care); failure in information gathering (eg, missed history or exam findings, ordering wrong tests, laboratory errors); failure in information interpretation (eg, exam finding or test result misinterpretation); inaccurate hypothesis generation (eg, due to suboptimal prioritization or weighing of supporting evidence); and failure in communication (eg, with other team members or with the patient).^2,34 Reasons for diagnostic process failures vary widely across different health care settings. While clinician assessment errors (eg, failure to consider or alternatively overweigh competing diagnoses) and errors in testing and the monitoring phase (eg, failure to order or follow up diagnostic tests) can lead to a majority of diagnostic errors in some patient populations, in other settings, social (eg, poor health literacy, punitive cultural practices) and economic factors (eg, lack of access to appropriate diagnostic tests or to specialty expertise) play a more prominent role.^34,35

The Figure describes the relationship between components of the diagnostic process and subsequent outcomes, including diagnostic process failures, diagnostic errors, and absence or presence of patient harm.^2,36,37 It reemphasizes the centrality of the patient in decision-making and the continuous nature of the process. The Figure also illustrates that only a minority of process failures result in diagnostic errors, and a smaller proportion of diagnostic errors actually lead to patient harm. Conversely, it also shows that diagnostic errors can happen without any obvious process-failure points, and, similarly, patient harm can take place in the absence of any evident diagnostic errors.^36-38 Finally, it highlights the need to incorporate feedback from process failures, diagnostic errors, and favorable and unfavorable patient outcomes in order to inform future quality improvement efforts and research.

A significant proportion of diagnostic errors are due to system-related vulnerabilities, such as limitations in availability, adoption or quality of work force training, health informatics resources, and diagnostic capabilities. Lack of institutional culture that promotes safety and transparency also predisposes to diagnostic errors.^39,40 The other major domain of process failures is related to cognitive errors in clinician decision-making. Anchoring, confirmation bias, availability bias, and base-rate neglect are some of the common cognitive biases that, along with personality traits (aversion to risk or ambiguity, overconfidence) and affective biases (influence of emotion on decision-making), often determine the degree of utilization of resources and the possibility of suboptimal diagnostic performance.^41,42 Further, implicit biases related to age, race, gender, and sexual orientation contribute to disparities in access to health care and outcomes.⁴³ In a large number of cases of preventable adverse outcomes, however, there are multiple interdependent individual and system-related failure points that lead to diagnostic error and patient harm.^6,32

Challenges in Defining and Measuring Diagnostic Errors

In order to develop effective, evidence-based interventions to reduce diagnostic errors in hospitalized patients, it is essential to be able to first operationally define, and then accurately measure, diagnostic errors and the process failures that contribute to these errors in a standardized way that is reproducible across different settings.^6,44 There are a number of obstacles in this endeavor.

A fundamental problem is that establishing a diagnosis is not a single act but a process. Patterns of symptoms and clinical presentations often differ for the same disease. Information required to make a diagnosis is usually gathered in stages, where the clinician obtains additional data, while considering many possibilities, of which 1 may be ultimately correct. Diagnoses evolve over time and in different care settings. “The most likely diagnosis” is not always the same as “the final correct diagnosis.” Moreover, the diagnostic process is influenced by patients’ individual clinical courses and preferences over time. This makes determination of missed, delayed, or incorrect diagnoses challenging.^45,46

For hospitalized patients, generally the goal is to first rule out more serious and acute conditions (eg, pulmonary embolism or stroke), even if their probability is rather low. Conversely, a diagnosis that appears less consequential if delayed (eg, chronic anemia of unclear etiology) might not be pursued on an urgent basis, and is often left to outpatient providers to examine, but still may manifest in downstream harm (eg, delayed diagnosis of gastrointestinal malignancy or recurrent admissions for heart failure due to missed iron-deficiency anemia). Therefore, coming up with disease diagnosis likelihoods in hindsight may turn out to be highly subjective and not always accurate. This can be particularly difficult when clinician and other team deliberations are not recorded in their entirety.⁴⁷

Another hurdle in the practice of diagnostic medicine is to preserve the balance between underdiagnosing versus pursuing overly aggressive diagnostic approaches. Conducting laboratory, imaging, or other diagnostic studies without a clear shared understanding of how they would affect clinical decision-making (eg, use of prostate-specific antigen to detect prostate cancer) not only leads to increased costs but can also delay appropriate care. Worse, subsequent unnecessary diagnostic tests and treatments can sometimes lead to serious harm.^48,49

Finally, retrospective reviews by clinicians are subject to multiple potential limitations that include failure to create well-defined research questions, poorly developed inclusion and exclusion criteria, and issues related to inter- and intra-rater reliability.⁵⁰ These methodological deficiencies can occur despite following "best practice" guidelines during the study planning, execution, and analysis phases. They further add to the challenge of defining and measuring diagnostic errors.⁴⁷

Strategies to Improve Measurement of Diagnostic Errors

Development of new methodologies to reliably measure diagnostic errors is an area of active research. The advancement of uniform and universally agreed-upon frameworks to define and identify process failure points and diagnostic errors would help reduce measurement error and support development and testing of interventions that could be generalizable across different health care settings. To more accurately define and measure diagnostic errors, several novel approaches have been proposed (Table 2).

The Safer Dx framework is an all-round tool developed to advance the discipline of measuring diagnostic errors. For an episode of care under review, the instrument scores various items to determine the likelihood of a diagnostic error. These items evaluate multiple dimensions affecting diagnostic performance and measurements across 3 broad domains: structure (provider and organizational characteristics—from everyone involved with patient care, to computing infrastructure, to policies and regulations), process (elements of the patient-provider encounter, diagnostic test performance and follow-up, and subspecialty- and referral-specific factors), and outcome (establishing accurate and timely diagnosis as opposed to missed, delayed, or incorrect diagnosis). This instrument has been revised and can be further modified by a variety of stakeholders, including clinicians, health care organizations, and policymakers, to identify potential diagnostic errors in a standardized way for patient safety and quality improvement research.^51,52

Use of standardized tools, such as the Diagnosis Error Evaluation and Research (DEER) taxonomy, can help to identify and classify specific failure points across different diagnostic process dimensions.³⁷ These failure points can be classified into: issues related to patient presentation or access to health care; failure to obtain or misinterpretation of history or physical exam findings; errors in use of diagnostics tests due to technical or clinician-related factors; failures in appropriate weighing of evidence and hypothesis generation; errors associated with referral or consultation process; and failure to monitor the patient or obtain timely follow-up.³⁴ The DEER taxonomy can also be modified based on specific research questions and study populations. Further, it can be recategorized to correspond to Safer Dx framework diagnostic process dimensions to provide insights into reasons for specific process failures and to develop new interventions to mitigate errors and patient harm.⁶

Since a majority of diagnostic errors do not lead to actual harm, use of “triggers” or clues (eg, procedure-related complications, patient falls, transfers to a higher level of care, readmissions within 30 days) can be a more efficient method to identify diagnostic errors and adverse events that do cause harm. The Global Trigger Tool, developed by the Institute for Healthcare Improvement, uses this strategy. This tool has been shown to identify a significantly higher number of serious adverse events than comparable methods.⁵³ This facilitates selection and development of strategies at the institutional level that are most likely to improve patient outcomes.²⁴

Encouraging and facilitating voluntary or prompted reporting from patients and clinicians can also play an important role in capturing diagnostic errors. Patients and clinicians are not only the key stakeholders but are also uniquely placed within the diagnostic process to detect and report potential errors.^25,54 Patient-safety-event reporting systems, such as RL6, play a vital role in reporting near-misses and adverse events. These systems provide a mechanism for team members at all levels within the hospital to contribute toward reporting patient adverse events, including those arising from diagnostic errors.⁵⁵ The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is the first standardized, nationally reported patient survey designed to measure patients’ perceptions of their hospital experience. The US Centers for Medicare and Medicaid Services (CMS) publishes HCAHPS results on its website 4 times a year, which serves as an important incentive for hospitals to improve patient safety and quality of health care delivery.⁵⁶

Another novel approach links multiple symptoms to a range of target diseases using the Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) framework. Using “big data” technologies, this technique can help discover otherwise hidden symptom-disease links and improve overall diagnostic performance. This approach is proposed for both case-control (look-back) and cohort (look-forward) studies assessing diagnostic errors and misdiagnosis-related harms. For example, starting with a known diagnosis with high potential for harm (eg, stroke), the “look-back” approach can be used to identify high-risk symptoms (eg, dizziness, vertigo). In the “look-forward” approach, a single symptom or exposure risk factor known to be frequently misdiagnosed (eg, dizziness) can be analyzed to identify potential adverse disease outcomes (eg, stroke, migraine).⁵⁷

Many large ongoing studies looking at diagnostic errors among hospitalized patients, such as Utility of Predictive Systems to identify Inpatient Diagnostic Errors (UPSIDE),⁵⁸Patient Safety Learning Lab (PSLL),⁵⁹ and Achieving Diagnostic Excellence through Prevention and Teamwork (ADEPT),⁶⁰ are using structured chart review methodologies incorporating many of the above strategies in combination. Cases triggered by certain events (eg, ICU transfer, death, rapid response event, new or worsening acute kidney injury) are reviewed using validated tools, including Safer Dx framework and DEER taxonomy, to provide the most precise estimates of the burden of diagnostic errors in hospitalized patients. These estimates may be much higher than previously predicted using traditional chart review approaches.^6,24 For example, a recently published study of 2809 random admissions in 11 Massachusetts hospitals identified 978 adverse events but only 10 diagnostic errors (diagnostic error rate, 0.4%).¹⁹ This was likely because the trigger method used in the study did not specifically examine the diagnostic process as critically as done by the Safer Dx framework and DEER taxonomy tools, thereby underestimating the total number of diagnostic errors. Further, these ongoing studies (eg, UPSIDE, ADEPT) aim to employ new and upcoming advanced machine-learning methods to create models that can improve overall diagnostic performance. This would pave the way to test and build novel, efficient, and scalable interventions to reduce diagnostic errors and improve patient outcomes.

Strategies to Improve Diagnostic Safety in Hospitalized Patients

Disease-specific biomedical research, as well as advances in laboratory, imaging, and other technologies, play a critical role in improving diagnostic accuracy. However, these technical approaches do not address many of the broader clinician- and system-level failure points and opportunities for improvement. Various patient-, provider-, and organizational-level interventions that could make diagnostic processes more resilient and reduce the risk of error and patient harm have been proposed.⁶¹

Among these strategies are approaches to empower patients and their families. Fostering therapeutic relationships between patients and members of the care team is essential to reducing diagnostic errors.⁶² Facilitating timely access to health records, ensuring transparency in decision making, and tailoring communication strategies to patients’ cultural and educational backgrounds can reduce harm.⁶³ Similarly, at the system level, enhancing communication among different providers by use of tools such as structured handoffs can prevent communication breakdowns and facilitate positive outcomes.⁶⁴

Interventions targeted at individual health care providers, such as educational programs to improve content-specific knowledge, can enhance diagnostic performance. Regular feedback, strategies to enhance equity, and fostering an environment where all providers are actively encouraged to think critically and participate in the diagnostic process (training programs to use “diagnostic time-outs” and making it a “team sport”) can improve clinical reasoning.^65,66 Use of standardized patients can help identify individual-level cognitive failure points and facilitate creation of new interventions to improve clinical decision-making processes.⁶⁷

Novel health information technologies can further augment these efforts. These include effective documentation by maintaining dynamic and accurate patient histories, problem lists, and medication lists^68-70; use of electronic health record–based algorithms to identify potential diagnostic delays for serious conditions^71,72; use of telemedicine technologies to improve accessibility and coordination⁷³;application of mobile health and wearable technologies to facilitate data-gathering and care delivery^74,75; and use of computerized decision-support tools, including applications to interpret electrocardiograms, imaging studies, and other diagnostic tests.⁷⁶

Use of precision medicine, powered by new artificial intelligence (AI) tools, is becoming more widespread. Algorithms powered by AI can augment and sometimes even outperform clinician decision-making in areas such as oncology, radiology, and primary care.⁷⁷ Creation of large biobanks like the All of Us research program can be used to study thousands of environmental and genetic risk factors and health conditions simultaneously, and help identify specific treatments that work best for people of different backgrounds.⁷⁸ Active research in these areas holds great promise in terms of how and when we diagnose diseases and make appropriate preventative and treatment decisions. Significant scientific, ethical, and regulatory challenges will need to be overcome before these technologies can address some of the most complex problems in health care.⁷⁹

Finally, diagnostic performance is affected by the external environment, including the functioning of the medical liability system. Diagnostic errors that lead to patient harm are a leading cause of malpractice claims.⁸⁰ Developing a legal environment, in collaboration with patient advocacy groups and health care organizations, that promotes and facilitates timely disclosure of diagnostic errors could decrease the incentive to hide errors, advance care processes, and improve outcomes.^81,82

Conclusion

The burden of diagnostic errors in hospitalized patients is unacceptably high and remains an underemphasized cause of preventable morbidity and mortality. Diagnostic errors often result from a breakdown in multiple interdependent processes that involve patient-, provider-, and system-level factors. Significant challenges remain in defining and identifying diagnostic errors as well as underlying process-failure points. The most effective interventions to reduce diagnostic errors will require greater patient participation in the diagnostic process and a mix of evidence-based interventions that promote individual-provider excellence as well as system-level changes. Further research and collaboration among various stakeholders should help improve diagnostic safety for hospitalized patients.

Corresponding author: Abhishek Goyal, MD, MPH; agoyal4@bwh.harvard.edu

Disclosures: Dr. Dalal disclosed receiving income ≥ $250 from MayaMD.

Abstract

Diagnostic errors in hospitalized patients are a leading cause of preventable morbidity and mortality. Significant challenges in defining and measuring diagnostic errors and underlying process failure points have led to considerable variability in reported rates of diagnostic errors and adverse outcomes. In this article, we explore the diagnostic process and its discrete components, emphasizing the centrality of the patient in decision-making as well as the continuous nature of the process. We review the incidence of diagnostic errors in hospitalized patients and different methodological approaches that have been used to arrive at these estimates. We discuss different but interdependent provider- and system-related process-failure points that lead to diagnostic errors. We examine specific challenges related to measurement of diagnostic errors and describe traditional and novel approaches that are being used to obtain the most precise estimates. Finally, we examine various patient-, provider-, and organizational-level interventions that have been proposed to improve diagnostic safety in hospitalized patients.

Keywords: diagnostic error, hospital medicine, patient safety.

Diagnosis is defined as a “pre-existing set of categories agreed upon by the medical profession to designate a specific condition.”¹ The diagnostic process involves obtaining a clinical history, performing a physical examination, conducting diagnostic testing, and consulting with other clinical providers to gather data that are relevant to understanding the underlying disease processes. This exercise involves generating hypotheses and updating prior probabilities as more information and evidence become available. Throughout this process of information gathering, integration, and interpretation, there is an ongoing assessment of whether sufficient and necessary knowledge has been obtained to make an accurate diagnosis and provide appropriate treatment.²

Diagnostic error is defined as a missed opportunity to make a timely diagnosis as part of this iterative process, including the failure of communicating the diagnosis to the patient in a timely manner.³ It can be categorized as a missed, delayed, or incorrect diagnosis based on available evidence at the time. Establishing the correct diagnosis has important implications. A timely and precise diagnosis ensures the patient the highest probability of having a positive health outcome that reflects an appropriate understanding of underlying disease processes and is consistent with their overall goals of care.³ When diagnostic errors occur, they can cause patient harm. Adverse events due to medical errors, including diagnostic errors, are estimated to be the third leading cause of death in the United States.⁴ Most people will experience at least 1 diagnostic error in their lifetime. In the 2015 National Academy of Medicine report Improving Diagnosis in Healthcare, diagnostic errors were identified as a major hazard as well as an opportunity to improve patient outcomes.²

Diagnostic errors during hospitalizations are especially concerning, as they are more likely to be implicated in a wider spectrum of harm, including permanent disability and death. This has become even more relevant for hospital medicine physicians and other clinical providers as they encounter increasing cognitive and administrative workloads, rising dissatisfaction and burnout, and unique obstacles such as night-time scheduling.⁵

Incidence of Diagnostic Errors in Hospitalized Patients

Several methodological approaches have been used to estimate the incidence of diagnostic errors in hospitalized patients. These include retrospective reviews of a sample of all hospital admissions, evaluations of selected adverse outcomes including autopsy studies, patient and provider surveys, and malpractice claims. Laboratory testing audits and secondary reviews in other diagnostic subspecialities (eg, radiology, pathology, and microbiology) are also essential to improving diagnostic performance in these specialized fields, which in turn affects overall hospital diagnostic error rates.^6-8 These diverse approaches provide unique insights regarding our ability to assess the degree to which potential harms, ranging from temporary impairment to permanent disability, to death, are attributable to different failure points in the diagnostic process.

Large retrospective chart reviews of random hospital admissions remain the most accurate way to determine the overall incidence of diagnostic errors in hospitalized patients.⁹ The Harvard Medical Practice Study, published in 1991, laid the groundwork for measuring the incidence of adverse events in hospitalized patients and assessing their relation to medical error, negligence, and disability. Reviewing 30,121 randomly selected records from 51 randomly selected acute care hospitals in New York State, the study found that adverse events occurred in 3.7% of hospitalizations, diagnostic errors accounted for 13.8% of these events, and these errors were likely attributable to negligence in 74.7% of cases. The study not only outlined individual-level process failures, but also focused attention on some of the systemic causes, setting the agenda for quality improvement research in hospital-based care for years to come.^10-12 A recent systematic review and meta-analysis of 22 hospital admission studies found a pooled rate of 0.7% (95% CI, 0.5%-1.1%) for harmful diagnostic errors.⁹ It found significant variations in the rates of adverse events, diagnostic errors, and range of diagnoses that were missed. This was primarily because of variabilities in pre-test probabilities in detecting diagnostic errors in these specific cohorts, as well as due to heterogeneity in study definitions and methodologies, especially regarding how they defined and measured “diagnostic error.” The analysis, however, did not account for diagnostic errors that were not related to patient harm (missed opportunities); therefore, it likely significantly underestimated the true incidence of diagnostic errors in these study populations. Table 1 summarizes some of key studies that have examined the incidence of harmful diagnostic errors in hospitalized patients.^9-21

The chief limitation of reviewing random hospital admissions is that, since overall rates of diagnostic errors are still relatively low, a large number of case reviews are required to identify a sufficient sample of adverse outcomes to gain a meaningful understanding of the underlying process failure points and develop tools for remediation. Patient and provider surveys or data from malpractice claims can be high-yield starting points for research on process errors.^22,23 Reviews of enriched cohorts of adverse outcomes, such as rapid-response events, intensive care unit (ICU) transfers, deaths, and hospital readmissions, can be an efficient way to identify process failures that lead to greatest harm. Depending on the research approach and the types of underlying patient populations sampled, rates of diagnostic errors in these high-risk groups have been estimated to be approximately 5% to 20%, or even higher.^6,24-31 For example, a retrospective study of 391 cases of unplanned 7-day readmissions found that 5.6% of cases contained at least 1 diagnostic error during the index admission.³² In a study conducted at 6 Belgian acute-care hospitals, 56% of patients requiring an unplanned transfer to a higher level of care were determined to have had an adverse event, and of these adverse events, 12.4% of cases were associated with errors in diagnosis.²⁹ A systematic review of 16 hospital-based studies estimated that 3.1% of all inpatient deaths were likely preventable, which corresponded to 22,165 deaths annually in the United States.³⁰ Another such review of 31 autopsy studies reported that 28% of autopsied ICU patients had at least 1 misdiagnosis; of these diagnostic errors, 8% were classified as potentially lethal, and 15% were considered major but not lethal.³¹ Significant drawbacks of such enriched cohort studies, however, are their poor generalizability and inability to detect failure points that do not lead to patient harm (near-miss events).³³

Causes of Diagnostic Errors in Hospitalized Patients

All aspects of the diagnostic process are susceptible to errors. These errors stem from a variety of faulty processes, including failure of the patient to engage with the health care system (eg, due to lack of insurance or transportation, or delay in seeking care); failure in information gathering (eg, missed history or exam findings, ordering wrong tests, laboratory errors); failure in information interpretation (eg, exam finding or test result misinterpretation); inaccurate hypothesis generation (eg, due to suboptimal prioritization or weighing of supporting evidence); and failure in communication (eg, with other team members or with the patient).^2,34 Reasons for diagnostic process failures vary widely across different health care settings. While clinician assessment errors (eg, failure to consider or alternatively overweigh competing diagnoses) and errors in testing and the monitoring phase (eg, failure to order or follow up diagnostic tests) can lead to a majority of diagnostic errors in some patient populations, in other settings, social (eg, poor health literacy, punitive cultural practices) and economic factors (eg, lack of access to appropriate diagnostic tests or to specialty expertise) play a more prominent role.^34,35

The Figure describes the relationship between components of the diagnostic process and subsequent outcomes, including diagnostic process failures, diagnostic errors, and absence or presence of patient harm.^2,36,37 It reemphasizes the centrality of the patient in decision-making and the continuous nature of the process. The Figure also illustrates that only a minority of process failures result in diagnostic errors, and a smaller proportion of diagnostic errors actually lead to patient harm. Conversely, it also shows that diagnostic errors can happen without any obvious process-failure points, and, similarly, patient harm can take place in the absence of any evident diagnostic errors.^36-38 Finally, it highlights the need to incorporate feedback from process failures, diagnostic errors, and favorable and unfavorable patient outcomes in order to inform future quality improvement efforts and research.

A significant proportion of diagnostic errors are due to system-related vulnerabilities, such as limitations in availability, adoption or quality of work force training, health informatics resources, and diagnostic capabilities. Lack of institutional culture that promotes safety and transparency also predisposes to diagnostic errors.^39,40 The other major domain of process failures is related to cognitive errors in clinician decision-making. Anchoring, confirmation bias, availability bias, and base-rate neglect are some of the common cognitive biases that, along with personality traits (aversion to risk or ambiguity, overconfidence) and affective biases (influence of emotion on decision-making), often determine the degree of utilization of resources and the possibility of suboptimal diagnostic performance.^41,42 Further, implicit biases related to age, race, gender, and sexual orientation contribute to disparities in access to health care and outcomes.⁴³ In a large number of cases of preventable adverse outcomes, however, there are multiple interdependent individual and system-related failure points that lead to diagnostic error and patient harm.^6,32

Challenges in Defining and Measuring Diagnostic Errors

In order to develop effective, evidence-based interventions to reduce diagnostic errors in hospitalized patients, it is essential to be able to first operationally define, and then accurately measure, diagnostic errors and the process failures that contribute to these errors in a standardized way that is reproducible across different settings.^6,44 There are a number of obstacles in this endeavor.

A fundamental problem is that establishing a diagnosis is not a single act but a process. Patterns of symptoms and clinical presentations often differ for the same disease. Information required to make a diagnosis is usually gathered in stages, where the clinician obtains additional data, while considering many possibilities, of which 1 may be ultimately correct. Diagnoses evolve over time and in different care settings. “The most likely diagnosis” is not always the same as “the final correct diagnosis.” Moreover, the diagnostic process is influenced by patients’ individual clinical courses and preferences over time. This makes determination of missed, delayed, or incorrect diagnoses challenging.^45,46

For hospitalized patients, generally the goal is to first rule out more serious and acute conditions (eg, pulmonary embolism or stroke), even if their probability is rather low. Conversely, a diagnosis that appears less consequential if delayed (eg, chronic anemia of unclear etiology) might not be pursued on an urgent basis, and is often left to outpatient providers to examine, but still may manifest in downstream harm (eg, delayed diagnosis of gastrointestinal malignancy or recurrent admissions for heart failure due to missed iron-deficiency anemia). Therefore, coming up with disease diagnosis likelihoods in hindsight may turn out to be highly subjective and not always accurate. This can be particularly difficult when clinician and other team deliberations are not recorded in their entirety.⁴⁷

Another hurdle in the practice of diagnostic medicine is to preserve the balance between underdiagnosing versus pursuing overly aggressive diagnostic approaches. Conducting laboratory, imaging, or other diagnostic studies without a clear shared understanding of how they would affect clinical decision-making (eg, use of prostate-specific antigen to detect prostate cancer) not only leads to increased costs but can also delay appropriate care. Worse, subsequent unnecessary diagnostic tests and treatments can sometimes lead to serious harm.^48,49

Finally, retrospective reviews by clinicians are subject to multiple potential limitations that include failure to create well-defined research questions, poorly developed inclusion and exclusion criteria, and issues related to inter- and intra-rater reliability.⁵⁰ These methodological deficiencies can occur despite following "best practice" guidelines during the study planning, execution, and analysis phases. They further add to the challenge of defining and measuring diagnostic errors.⁴⁷

Strategies to Improve Measurement of Diagnostic Errors

Development of new methodologies to reliably measure diagnostic errors is an area of active research. The advancement of uniform and universally agreed-upon frameworks to define and identify process failure points and diagnostic errors would help reduce measurement error and support development and testing of interventions that could be generalizable across different health care settings. To more accurately define and measure diagnostic errors, several novel approaches have been proposed (Table 2).

The Safer Dx framework is an all-round tool developed to advance the discipline of measuring diagnostic errors. For an episode of care under review, the instrument scores various items to determine the likelihood of a diagnostic error. These items evaluate multiple dimensions affecting diagnostic performance and measurements across 3 broad domains: structure (provider and organizational characteristics—from everyone involved with patient care, to computing infrastructure, to policies and regulations), process (elements of the patient-provider encounter, diagnostic test performance and follow-up, and subspecialty- and referral-specific factors), and outcome (establishing accurate and timely diagnosis as opposed to missed, delayed, or incorrect diagnosis). This instrument has been revised and can be further modified by a variety of stakeholders, including clinicians, health care organizations, and policymakers, to identify potential diagnostic errors in a standardized way for patient safety and quality improvement research.^51,52

Use of standardized tools, such as the Diagnosis Error Evaluation and Research (DEER) taxonomy, can help to identify and classify specific failure points across different diagnostic process dimensions.³⁷ These failure points can be classified into: issues related to patient presentation or access to health care; failure to obtain or misinterpretation of history or physical exam findings; errors in use of diagnostics tests due to technical or clinician-related factors; failures in appropriate weighing of evidence and hypothesis generation; errors associated with referral or consultation process; and failure to monitor the patient or obtain timely follow-up.³⁴ The DEER taxonomy can also be modified based on specific research questions and study populations. Further, it can be recategorized to correspond to Safer Dx framework diagnostic process dimensions to provide insights into reasons for specific process failures and to develop new interventions to mitigate errors and patient harm.⁶

Since a majority of diagnostic errors do not lead to actual harm, use of “triggers” or clues (eg, procedure-related complications, patient falls, transfers to a higher level of care, readmissions within 30 days) can be a more efficient method to identify diagnostic errors and adverse events that do cause harm. The Global Trigger Tool, developed by the Institute for Healthcare Improvement, uses this strategy. This tool has been shown to identify a significantly higher number of serious adverse events than comparable methods.⁵³ This facilitates selection and development of strategies at the institutional level that are most likely to improve patient outcomes.²⁴

Encouraging and facilitating voluntary or prompted reporting from patients and clinicians can also play an important role in capturing diagnostic errors. Patients and clinicians are not only the key stakeholders but are also uniquely placed within the diagnostic process to detect and report potential errors.^25,54 Patient-safety-event reporting systems, such as RL6, play a vital role in reporting near-misses and adverse events. These systems provide a mechanism for team members at all levels within the hospital to contribute toward reporting patient adverse events, including those arising from diagnostic errors.⁵⁵ The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey is the first standardized, nationally reported patient survey designed to measure patients’ perceptions of their hospital experience. The US Centers for Medicare and Medicaid Services (CMS) publishes HCAHPS results on its website 4 times a year, which serves as an important incentive for hospitals to improve patient safety and quality of health care delivery.⁵⁶

Another novel approach links multiple symptoms to a range of target diseases using the Symptom-Disease Pair Analysis of Diagnostic Error (SPADE) framework. Using “big data” technologies, this technique can help discover otherwise hidden symptom-disease links and improve overall diagnostic performance. This approach is proposed for both case-control (look-back) and cohort (look-forward) studies assessing diagnostic errors and misdiagnosis-related harms. For example, starting with a known diagnosis with high potential for harm (eg, stroke), the “look-back” approach can be used to identify high-risk symptoms (eg, dizziness, vertigo). In the “look-forward” approach, a single symptom or exposure risk factor known to be frequently misdiagnosed (eg, dizziness) can be analyzed to identify potential adverse disease outcomes (eg, stroke, migraine).⁵⁷

Many large ongoing studies looking at diagnostic errors among hospitalized patients, such as Utility of Predictive Systems to identify Inpatient Diagnostic Errors (UPSIDE),⁵⁸Patient Safety Learning Lab (PSLL),⁵⁹ and Achieving Diagnostic Excellence through Prevention and Teamwork (ADEPT),⁶⁰ are using structured chart review methodologies incorporating many of the above strategies in combination. Cases triggered by certain events (eg, ICU transfer, death, rapid response event, new or worsening acute kidney injury) are reviewed using validated tools, including Safer Dx framework and DEER taxonomy, to provide the most precise estimates of the burden of diagnostic errors in hospitalized patients. These estimates may be much higher than previously predicted using traditional chart review approaches.^6,24 For example, a recently published study of 2809 random admissions in 11 Massachusetts hospitals identified 978 adverse events but only 10 diagnostic errors (diagnostic error rate, 0.4%).¹⁹ This was likely because the trigger method used in the study did not specifically examine the diagnostic process as critically as done by the Safer Dx framework and DEER taxonomy tools, thereby underestimating the total number of diagnostic errors. Further, these ongoing studies (eg, UPSIDE, ADEPT) aim to employ new and upcoming advanced machine-learning methods to create models that can improve overall diagnostic performance. This would pave the way to test and build novel, efficient, and scalable interventions to reduce diagnostic errors and improve patient outcomes.

Strategies to Improve Diagnostic Safety in Hospitalized Patients

Disease-specific biomedical research, as well as advances in laboratory, imaging, and other technologies, play a critical role in improving diagnostic accuracy. However, these technical approaches do not address many of the broader clinician- and system-level failure points and opportunities for improvement. Various patient-, provider-, and organizational-level interventions that could make diagnostic processes more resilient and reduce the risk of error and patient harm have been proposed.⁶¹

Among these strategies are approaches to empower patients and their families. Fostering therapeutic relationships between patients and members of the care team is essential to reducing diagnostic errors.⁶² Facilitating timely access to health records, ensuring transparency in decision making, and tailoring communication strategies to patients’ cultural and educational backgrounds can reduce harm.⁶³ Similarly, at the system level, enhancing communication among different providers by use of tools such as structured handoffs can prevent communication breakdowns and facilitate positive outcomes.⁶⁴

Interventions targeted at individual health care providers, such as educational programs to improve content-specific knowledge, can enhance diagnostic performance. Regular feedback, strategies to enhance equity, and fostering an environment where all providers are actively encouraged to think critically and participate in the diagnostic process (training programs to use “diagnostic time-outs” and making it a “team sport”) can improve clinical reasoning.^65,66 Use of standardized patients can help identify individual-level cognitive failure points and facilitate creation of new interventions to improve clinical decision-making processes.⁶⁷

Novel health information technologies can further augment these efforts. These include effective documentation by maintaining dynamic and accurate patient histories, problem lists, and medication lists^68-70; use of electronic health record–based algorithms to identify potential diagnostic delays for serious conditions^71,72; use of telemedicine technologies to improve accessibility and coordination⁷³;application of mobile health and wearable technologies to facilitate data-gathering and care delivery^74,75; and use of computerized decision-support tools, including applications to interpret electrocardiograms, imaging studies, and other diagnostic tests.⁷⁶

Use of precision medicine, powered by new artificial intelligence (AI) tools, is becoming more widespread. Algorithms powered by AI can augment and sometimes even outperform clinician decision-making in areas such as oncology, radiology, and primary care.⁷⁷ Creation of large biobanks like the All of Us research program can be used to study thousands of environmental and genetic risk factors and health conditions simultaneously, and help identify specific treatments that work best for people of different backgrounds.⁷⁸ Active research in these areas holds great promise in terms of how and when we diagnose diseases and make appropriate preventative and treatment decisions. Significant scientific, ethical, and regulatory challenges will need to be overcome before these technologies can address some of the most complex problems in health care.⁷⁹

Finally, diagnostic performance is affected by the external environment, including the functioning of the medical liability system. Diagnostic errors that lead to patient harm are a leading cause of malpractice claims.⁸⁰ Developing a legal environment, in collaboration with patient advocacy groups and health care organizations, that promotes and facilitates timely disclosure of diagnostic errors could decrease the incentive to hide errors, advance care processes, and improve outcomes.^81,82

Conclusion

The burden of diagnostic errors in hospitalized patients is unacceptably high and remains an underemphasized cause of preventable morbidity and mortality. Diagnostic errors often result from a breakdown in multiple interdependent processes that involve patient-, provider-, and system-level factors. Significant challenges remain in defining and identifying diagnostic errors as well as underlying process-failure points. The most effective interventions to reduce diagnostic errors will require greater patient participation in the diagnostic process and a mix of evidence-based interventions that promote individual-provider excellence as well as system-level changes. Further research and collaboration among various stakeholders should help improve diagnostic safety for hospitalized patients.

Corresponding author: Abhishek Goyal, MD, MPH; agoyal4@bwh.harvard.edu

Disclosures: Dr. Dalal disclosed receiving income ≥ $250 from MayaMD.

Each year, 40,000 to 98,000 deaths occur due to medical errors.¹ The Harvard Medical Practice Study (HMPS), published in 1991, found that 3.7% of hospitalized patients were harmed by adverse events and 1% were harmed by adverse events due to negligence.² The latest HMPS showed that, despite significant improvements in patient safety over the past 3 decades, patient safety challenges persist. This study found that inpatient care leads to harm in nearly a quarter of patients, and that 1 in 4 of these adverse events are preventable.³

Since the first HMPS study was published, efforts to improve patient safety have focused on identifying causes of medical error and the design and implementation of interventions to mitigate errors. Factors contributing to medical errors have been well documented: the complexity of care delivery from inpatient to outpatient settings, with transitions of care and extensive use of medications; multiple comorbidities; and the fragmentation of care across multiple systems and specialties. Although most errors are related to process or system failure, accountability of each practitioner and clinician is essential to promoting a culture of safety. Many medical errors are preventable through multifaceted approaches employed throughout the phases of the care,⁴ with medication errors, both prescribing and administration, and diagnostic and treatment errors encompassing most risk prevention areas. Broadly, safety efforts should emphasize building a culture of safety where all safety events are reported, including near-miss events.

Two articles in this issue of JCOM address key elements of patient safety: building a safety culture and diagnostic error. Merchant et al⁵ report on an initiative designed to promote a safety culture by recognizing and rewarding staff who identify and report near misses. The tiered awards program they designed led to significantly increased staff participation in the safety awards nomination process and was associated with increased reporting of actual and close-call events and greater attendance at monthly safety forums. Goyal et al,⁶ noting that diagnostic error rates in hospitalized patients remain unacceptably high, provide a concise update on diagnostic error among inpatients, focusing on issues related to defining and measuring diagnostic errors and current strategies to improve diagnostic safety in hospitalized patients. In a third article, Sathi et al report on efforts to teach quality improvement (QI) methods to internal medicine trainees; their project increased residents’ knowledge of their patient panels and comfort with QI approaches and led to improved patient outcomes.

Major progress has been made to improve health care safety since the first HMPS was published. However, the latest HMPS shows that patient safety efforts must continue, given the persistent risk for patient harm in the current health care delivery system. Safety, along with clear accountability for identifying, reporting, and addressing errors, should be a top priority for health care systems throughout the preventive, diagnostic, and therapeutic phases of care.

Corresponding author: Ebrahim Barkoudah, MD, MPH; ebarkoudah@bwh.harvard.edu

Each year, 40,000 to 98,000 deaths occur due to medical errors.¹ The Harvard Medical Practice Study (HMPS), published in 1991, found that 3.7% of hospitalized patients were harmed by adverse events and 1% were harmed by adverse events due to negligence.² The latest HMPS showed that, despite significant improvements in patient safety over the past 3 decades, patient safety challenges persist. This study found that inpatient care leads to harm in nearly a quarter of patients, and that 1 in 4 of these adverse events are preventable.³

Since the first HMPS study was published, efforts to improve patient safety have focused on identifying causes of medical error and the design and implementation of interventions to mitigate errors. Factors contributing to medical errors have been well documented: the complexity of care delivery from inpatient to outpatient settings, with transitions of care and extensive use of medications; multiple comorbidities; and the fragmentation of care across multiple systems and specialties. Although most errors are related to process or system failure, accountability of each practitioner and clinician is essential to promoting a culture of safety. Many medical errors are preventable through multifaceted approaches employed throughout the phases of the care,⁴ with medication errors, both prescribing and administration, and diagnostic and treatment errors encompassing most risk prevention areas. Broadly, safety efforts should emphasize building a culture of safety where all safety events are reported, including near-miss events.

Two articles in this issue of JCOM address key elements of patient safety: building a safety culture and diagnostic error. Merchant et al⁵ report on an initiative designed to promote a safety culture by recognizing and rewarding staff who identify and report near misses. The tiered awards program they designed led to significantly increased staff participation in the safety awards nomination process and was associated with increased reporting of actual and close-call events and greater attendance at monthly safety forums. Goyal et al,⁶ noting that diagnostic error rates in hospitalized patients remain unacceptably high, provide a concise update on diagnostic error among inpatients, focusing on issues related to defining and measuring diagnostic errors and current strategies to improve diagnostic safety in hospitalized patients. In a third article, Sathi et al report on efforts to teach quality improvement (QI) methods to internal medicine trainees; their project increased residents’ knowledge of their patient panels and comfort with QI approaches and led to improved patient outcomes.

Major progress has been made to improve health care safety since the first HMPS was published. However, the latest HMPS shows that patient safety efforts must continue, given the persistent risk for patient harm in the current health care delivery system. Safety, along with clear accountability for identifying, reporting, and addressing errors, should be a top priority for health care systems throughout the preventive, diagnostic, and therapeutic phases of care.

Corresponding author: Ebrahim Barkoudah, MD, MPH; ebarkoudah@bwh.harvard.edu

Each year, 40,000 to 98,000 deaths occur due to medical errors.¹ The Harvard Medical Practice Study (HMPS), published in 1991, found that 3.7% of hospitalized patients were harmed by adverse events and 1% were harmed by adverse events due to negligence.² The latest HMPS showed that, despite significant improvements in patient safety over the past 3 decades, patient safety challenges persist. This study found that inpatient care leads to harm in nearly a quarter of patients, and that 1 in 4 of these adverse events are preventable.³

Since the first HMPS study was published, efforts to improve patient safety have focused on identifying causes of medical error and the design and implementation of interventions to mitigate errors. Factors contributing to medical errors have been well documented: the complexity of care delivery from inpatient to outpatient settings, with transitions of care and extensive use of medications; multiple comorbidities; and the fragmentation of care across multiple systems and specialties. Although most errors are related to process or system failure, accountability of each practitioner and clinician is essential to promoting a culture of safety. Many medical errors are preventable through multifaceted approaches employed throughout the phases of the care,⁴ with medication errors, both prescribing and administration, and diagnostic and treatment errors encompassing most risk prevention areas. Broadly, safety efforts should emphasize building a culture of safety where all safety events are reported, including near-miss events.

Two articles in this issue of JCOM address key elements of patient safety: building a safety culture and diagnostic error. Merchant et al⁵ report on an initiative designed to promote a safety culture by recognizing and rewarding staff who identify and report near misses. The tiered awards program they designed led to significantly increased staff participation in the safety awards nomination process and was associated with increased reporting of actual and close-call events and greater attendance at monthly safety forums. Goyal et al,⁶ noting that diagnostic error rates in hospitalized patients remain unacceptably high, provide a concise update on diagnostic error among inpatients, focusing on issues related to defining and measuring diagnostic errors and current strategies to improve diagnostic safety in hospitalized patients. In a third article, Sathi et al report on efforts to teach quality improvement (QI) methods to internal medicine trainees; their project increased residents’ knowledge of their patient panels and comfort with QI approaches and led to improved patient outcomes.

Major progress has been made to improve health care safety since the first HMPS was published. However, the latest HMPS shows that patient safety efforts must continue, given the persistent risk for patient harm in the current health care delivery system. Safety, along with clear accountability for identifying, reporting, and addressing errors, should be a top priority for health care systems throughout the preventive, diagnostic, and therapeutic phases of care.

Corresponding author: Ebrahim Barkoudah, MD, MPH; ebarkoudah@bwh.harvard.edu

Study 1 Overview (Chang et al)

Objective: To assess the incidence of postoperative delirium (POD) following propofol- vs sevoflurane-based anesthesia in geriatric spine surgery patients.

Design: Retrospective, single-blinded observational study of propofol- and sevoflurane-based anesthesia cohorts.

Setting and participants: Patients eligible for this study were aged 65 years or older admitted to the SMG-SNU Boramae Medical Center (Seoul, South Korea). All patients underwent general anesthesia either via intravenous propofol or inhalational sevoflurane for spine surgery between January 2015 and December 2019. Patients were retrospectively identified via electronic medical records. Patient exclusion criteria included preoperative delirium, history of dementia, psychiatric disease, alcoholism, hepatic or renal dysfunction, postoperative mechanical ventilation dependence, other surgery within the recent 6 months, maintenance of intraoperative anesthesia with combined anesthetics, or incomplete medical record.

Main outcome measures: The primary outcome was the incidence of POD after administration of propofol- and sevoflurane-based anesthesia during hospitalization. Patients were screened for POD regularly by attending nurses using the Nursing Delirium Screening Scale (disorientation, inappropriate behavior, inappropriate communication, hallucination, and psychomotor retardation) during the entirety of the patient’s hospital stay; if 1 or more screening criteria were met, a psychiatrist was consulted for the proper diagnosis and management of delirium. A psychiatric diagnosis was required for a case to be counted toward the incidence of POD in this study. Secondary outcomes included postoperative 30-day complications (angina, myocardial infarction, transient ischemic attack/stroke, pneumonia, deep vein thrombosis, pulmonary embolism, acute kidney injury, or infection) and length of postoperative hospital stay.

Main results: POD occurred in 29 patients (10.3%) out of the total cohort of 281. POD was more common in the sevoflurane group than in the propofol group (15.7% vs 5.0%; P = .003). Using multivariable logistic regression, inhalational sevoflurane was associated with an increased risk of POD as compared to propofol-based anesthesia (odds ratio [OR], 4.120; 95% CI, 1.549-10.954; P = .005). There was no association between choice of anesthetic and postoperative 30-day complications or the length of postoperative hospital stay. Both older age (OR, 1.242; 95% CI, 1.130-1.366; P < .001) and higher pain score at postoperative day 1 (OR, 1.338; 95% CI, 1.056-1.696; P = .016) were associated with increased risk of POD.

Conclusion: Propofol-based anesthesia was associated with a lower incidence of and risk for POD than sevoflurane-based anesthesia in older patients undergoing spine surgery.

Study 2 Overview (Mei et al)

Objective: To determine the incidence and duration of POD in older patients after total knee/hip replacement (TKR/THR) under intravenous propofol or inhalational sevoflurane general anesthesia.

Design: Randomized clinical trial of propofol and sevoflurane groups.

Setting and participants: This study was conducted at the Shanghai Tenth People’s Hospital and involved 209 participants enrolled between June 2016 and November 2019. All participants were 60 years of age or older, scheduled for TKR/THR surgery under general anesthesia, American Society of Anesthesiologists (ASA) class I to III, and assessed to be of normal cognitive function preoperatively via a Mini-Mental State Examination. Participant exclusion criteria included preexisting delirium as assessed by the Confusion Assessment Method (CAM), prior diagnosed neurological diseases (eg, Parkinson’s disease), prior diagnosed mental disorders (eg, schizophrenia), or impaired vision or hearing that would influence cognitive assessments. All participants were randomly assigned to either sevoflurane or propofol anesthesia for their surgery via a computer-generated list. Of these, 103 received inhalational sevoflurane and 106 received intravenous propofol. All participants received standardized postoperative care.

Main outcome measures: All participants were interviewed by investigators, who were blinded to the anesthesia regimen, twice daily on postoperative days 1, 2, and 3 using CAM and a CAM-based scoring system (CAM-S) to assess delirium severity. The CAM encapsulated 4 criteria: acute onset and fluctuating course, agitation, disorganized thinking, and altered level of consciousness. To diagnose delirium, both the first and second criteria must be met, in addition to either the third or fourth criterion. The averages of the scores across the 3 postoperative days indicated delirium severity, while the incidence and duration of delirium was assessed by the presence of delirium as determined by CAM on any postoperative day.

Main results: All eligible participants (N = 209; mean [SD] age 71.2 [6.7] years; 29.2% male) were included in the final analysis. The incidence of POD was not statistically different between the propofol and sevoflurane groups (33.0% vs 23.3%; P = .119, Chi-square test). It was estimated that 316 participants in each arm of the study were needed to detect statistical differences. The number of days of POD per person were higher with propofol anesthesia as compared to sevoflurane (0.5 [0.8] vs 0.3 [0.5]; P =  .049, Student’s t-test).

Conclusion: This underpowered study showed a 9.7% difference in the incidence of POD between older adults who received propofol (33.0%) and sevoflurane (23.3%) after THR/TKR. Further studies with a larger sample size are needed to compare general anesthetics and their role in POD.

Commentary

Delirium is characterized by an acute state of confusion with fluctuating mental status, inattention, disorganized thinking, and altered level of consciousness. It is often caused by medications and/or their related adverse effects, infections, electrolyte imbalances, and other clinical etiologies. Delirium often manifests in post-surgical settings, disproportionately affecting older patients and leading to increased risk of morbidity, mortality, hospital length of stay, and health care costs.¹ Intraoperative risk factors for POD are determined by the degree of operative stress (eg, lower-risk surgeries put the patient at reduced risk for POD as compared to higher-risk surgeries) and are additive to preexisting patient-specific risk factors, such as older age and functional impairment.¹ Because operative stress is associated with risk for POD, limiting operative stress in controlled ways, such as through the choice of anesthetic agent administered, may be a pragmatic way to manage operative risks and optimize outcomes, especially when serving a surgically vulnerable population.

In Study 1, Chang et al sought to assess whether 2 commonly utilized general anesthetics, propofol and sevoflurane, in older patients undergoing spine surgery differentially affected the incidence of POD. In this retrospective, single-blinded observational study of 281 geriatric patients, the researchers found that sevoflurane was associated with a higher risk of POD as compared to propofol. However, these anesthetics were not associated with surgical outcomes such as postoperative 30-day complications or the length of postoperative hospital stay. While these findings added new knowledge to this field of research, several limitations should be kept in mind when interpreting this study’s results. For instance, the sample size was relatively small, with all cases selected from a single center utilizing a retrospective analysis. In addition, although a standardized nursing screening tool was used as a method for delirium detection, hypoactive delirium or less symptomatic delirium may have been missed, which in turn would lead to an underestimation of POD incidence. The latter is a common limitation in delirium research.

In Study 2, Mei et al similarly explored the effects of general anesthetics on POD in older surgical patients. Specifically, using a randomized clinical trial design, the investigators compared propofol with sevoflurane in older patients who underwent TKR/THR, and their roles in POD severity and duration. Although the incidence of POD was higher in those who received propofol compared to sevoflurane, this trial was underpowered and the results did not reach statistical significance. In addition, while the duration of POD was slightly longer in the propofol group compared to the sevoflurane group (0.5 vs 0.3 days), it was unclear if this finding was clinically significant. Similar to many research studies in POD, limitations of Study 2 included a small sample size of 209 patients, with all participants enrolled from a single center. On the other hand, this study illustrated the feasibility of a method that allowed reproducible prospective assessment of POD time course using CAM and CAM-S.

Applications for Clinical Practice and System Implementation

The delineation of risk factors that contribute to delirium after surgery in older patients is key to mitigating risks for POD and improving clinical outcomes. An important step towards a better understanding of these modifiable risk factors is to clearly quantify intraoperative risk of POD attributable to specific anesthetics. While preclinical studies have shown differential neurotoxicity effects of propofol and sevoflurane, their impact on clinically important neurologic outcomes such as delirium and cognitive decline remains poorly understood. Although Studies 1 and 2 both provided head-to-head comparisons of propofol and sevoflurane as risk factors for POD in high-operative-stress surgeries in older patients, the results were inconsistent. That being said, this small incremental increase in knowledge was not unexpected in the course of discovery around a clinically complex research question. Importantly, these studies provided evidence regarding the methodological approaches that could be taken to further this line of research.

The mediating factors of the differences on neurologic outcomes between anesthetic agents are likely pharmacological, biological, and methodological. Pharmacologically, the differences between target receptors, such as GABAA (propofol, etomidate) or NMDA (ketamine), could be a defining feature in the difference in incidence of POD. Additionally, secondary actions of anesthetic agents on glycine, nicotinic, and acetylcholine receptors could play a role as well. Biologically, genes such as CYP2E1, CYP2B6, CYP2C9, GSTP1, UGT1A9, SULT1A1, and NQO1 have all been identified as genetic factors in the metabolism of anesthetics, and variations in such genes could result in different responses to anesthetics.² Methodologically, routes of anesthetic administration (eg, inhalation vs intravenous), preexisting anatomical structures, or confounding medical conditions (eg, lower respiratory volume due to older age) may influence POD incidence, duration, or severity. Moreover, methodological differences between Studies 1 and 2, such as surgeries performed (spinal vs TKR/THR), patient populations (South Korean vs Chinese), and the diagnosis and monitoring of delirium (retrospective screening and diagnosis vs prospective CAM/CAM-S) may impact delirium outcomes. Thus, these factors should be considered in the design of future clinical trials undertaken to investigate the effects of anesthetics on POD.

Given the high prevalence of delirium and its associated adverse outcomes in the immediate postoperative period in older patients, further research is warranted to determine how anesthetics affect POD in order to optimize perioperative care and mitigate risks in this vulnerable population. Moreover, parallel investigations into how anesthetics differentially impact the development of transient or longer-term cognitive impairment after a surgical procedure (ie, postoperative cognitive dysfunction) in older adults are urgently needed in order to improve their cognitive health.

Practice Points

• Intravenous propofol and inhalational sevoflurane may be differentially associated with incidence, duration, and severity of POD in geriatric surgical patients.
• Further larger-scale studies are warranted to clarify the role of anesthetic choice in POD in order to optimize surgical outcomes in older patients.

–Jared Doan, BS, and Fred Ko, MD
Icahn School of Medicine at Mount Sinai

Study 1 Overview (Chang et al)

Objective: To assess the incidence of postoperative delirium (POD) following propofol- vs sevoflurane-based anesthesia in geriatric spine surgery patients.

Design: Retrospective, single-blinded observational study of propofol- and sevoflurane-based anesthesia cohorts.

Setting and participants: Patients eligible for this study were aged 65 years or older admitted to the SMG-SNU Boramae Medical Center (Seoul, South Korea). All patients underwent general anesthesia either via intravenous propofol or inhalational sevoflurane for spine surgery between January 2015 and December 2019. Patients were retrospectively identified via electronic medical records. Patient exclusion criteria included preoperative delirium, history of dementia, psychiatric disease, alcoholism, hepatic or renal dysfunction, postoperative mechanical ventilation dependence, other surgery within the recent 6 months, maintenance of intraoperative anesthesia with combined anesthetics, or incomplete medical record.

Main outcome measures: The primary outcome was the incidence of POD after administration of propofol- and sevoflurane-based anesthesia during hospitalization. Patients were screened for POD regularly by attending nurses using the Nursing Delirium Screening Scale (disorientation, inappropriate behavior, inappropriate communication, hallucination, and psychomotor retardation) during the entirety of the patient’s hospital stay; if 1 or more screening criteria were met, a psychiatrist was consulted for the proper diagnosis and management of delirium. A psychiatric diagnosis was required for a case to be counted toward the incidence of POD in this study. Secondary outcomes included postoperative 30-day complications (angina, myocardial infarction, transient ischemic attack/stroke, pneumonia, deep vein thrombosis, pulmonary embolism, acute kidney injury, or infection) and length of postoperative hospital stay.

Main results: POD occurred in 29 patients (10.3%) out of the total cohort of 281. POD was more common in the sevoflurane group than in the propofol group (15.7% vs 5.0%; P = .003). Using multivariable logistic regression, inhalational sevoflurane was associated with an increased risk of POD as compared to propofol-based anesthesia (odds ratio [OR], 4.120; 95% CI, 1.549-10.954; P = .005). There was no association between choice of anesthetic and postoperative 30-day complications or the length of postoperative hospital stay. Both older age (OR, 1.242; 95% CI, 1.130-1.366; P < .001) and higher pain score at postoperative day 1 (OR, 1.338; 95% CI, 1.056-1.696; P = .016) were associated with increased risk of POD.

Conclusion: Propofol-based anesthesia was associated with a lower incidence of and risk for POD than sevoflurane-based anesthesia in older patients undergoing spine surgery.

Study 2 Overview (Mei et al)

Objective: To determine the incidence and duration of POD in older patients after total knee/hip replacement (TKR/THR) under intravenous propofol or inhalational sevoflurane general anesthesia.

Design: Randomized clinical trial of propofol and sevoflurane groups.

Setting and participants: This study was conducted at the Shanghai Tenth People’s Hospital and involved 209 participants enrolled between June 2016 and November 2019. All participants were 60 years of age or older, scheduled for TKR/THR surgery under general anesthesia, American Society of Anesthesiologists (ASA) class I to III, and assessed to be of normal cognitive function preoperatively via a Mini-Mental State Examination. Participant exclusion criteria included preexisting delirium as assessed by the Confusion Assessment Method (CAM), prior diagnosed neurological diseases (eg, Parkinson’s disease), prior diagnosed mental disorders (eg, schizophrenia), or impaired vision or hearing that would influence cognitive assessments. All participants were randomly assigned to either sevoflurane or propofol anesthesia for their surgery via a computer-generated list. Of these, 103 received inhalational sevoflurane and 106 received intravenous propofol. All participants received standardized postoperative care.

Main outcome measures: All participants were interviewed by investigators, who were blinded to the anesthesia regimen, twice daily on postoperative days 1, 2, and 3 using CAM and a CAM-based scoring system (CAM-S) to assess delirium severity. The CAM encapsulated 4 criteria: acute onset and fluctuating course, agitation, disorganized thinking, and altered level of consciousness. To diagnose delirium, both the first and second criteria must be met, in addition to either the third or fourth criterion. The averages of the scores across the 3 postoperative days indicated delirium severity, while the incidence and duration of delirium was assessed by the presence of delirium as determined by CAM on any postoperative day.

Main results: All eligible participants (N = 209; mean [SD] age 71.2 [6.7] years; 29.2% male) were included in the final analysis. The incidence of POD was not statistically different between the propofol and sevoflurane groups (33.0% vs 23.3%; P = .119, Chi-square test). It was estimated that 316 participants in each arm of the study were needed to detect statistical differences. The number of days of POD per person were higher with propofol anesthesia as compared to sevoflurane (0.5 [0.8] vs 0.3 [0.5]; P =  .049, Student’s t-test).

Conclusion: This underpowered study showed a 9.7% difference in the incidence of POD between older adults who received propofol (33.0%) and sevoflurane (23.3%) after THR/TKR. Further studies with a larger sample size are needed to compare general anesthetics and their role in POD.

Commentary

Delirium is characterized by an acute state of confusion with fluctuating mental status, inattention, disorganized thinking, and altered level of consciousness. It is often caused by medications and/or their related adverse effects, infections, electrolyte imbalances, and other clinical etiologies. Delirium often manifests in post-surgical settings, disproportionately affecting older patients and leading to increased risk of morbidity, mortality, hospital length of stay, and health care costs.¹ Intraoperative risk factors for POD are determined by the degree of operative stress (eg, lower-risk surgeries put the patient at reduced risk for POD as compared to higher-risk surgeries) and are additive to preexisting patient-specific risk factors, such as older age and functional impairment.¹ Because operative stress is associated with risk for POD, limiting operative stress in controlled ways, such as through the choice of anesthetic agent administered, may be a pragmatic way to manage operative risks and optimize outcomes, especially when serving a surgically vulnerable population.

In Study 1, Chang et al sought to assess whether 2 commonly utilized general anesthetics, propofol and sevoflurane, in older patients undergoing spine surgery differentially affected the incidence of POD. In this retrospective, single-blinded observational study of 281 geriatric patients, the researchers found that sevoflurane was associated with a higher risk of POD as compared to propofol. However, these anesthetics were not associated with surgical outcomes such as postoperative 30-day complications or the length of postoperative hospital stay. While these findings added new knowledge to this field of research, several limitations should be kept in mind when interpreting this study’s results. For instance, the sample size was relatively small, with all cases selected from a single center utilizing a retrospective analysis. In addition, although a standardized nursing screening tool was used as a method for delirium detection, hypoactive delirium or less symptomatic delirium may have been missed, which in turn would lead to an underestimation of POD incidence. The latter is a common limitation in delirium research.

In Study 2, Mei et al similarly explored the effects of general anesthetics on POD in older surgical patients. Specifically, using a randomized clinical trial design, the investigators compared propofol with sevoflurane in older patients who underwent TKR/THR, and their roles in POD severity and duration. Although the incidence of POD was higher in those who received propofol compared to sevoflurane, this trial was underpowered and the results did not reach statistical significance. In addition, while the duration of POD was slightly longer in the propofol group compared to the sevoflurane group (0.5 vs 0.3 days), it was unclear if this finding was clinically significant. Similar to many research studies in POD, limitations of Study 2 included a small sample size of 209 patients, with all participants enrolled from a single center. On the other hand, this study illustrated the feasibility of a method that allowed reproducible prospective assessment of POD time course using CAM and CAM-S.

Applications for Clinical Practice and System Implementation

The delineation of risk factors that contribute to delirium after surgery in older patients is key to mitigating risks for POD and improving clinical outcomes. An important step towards a better understanding of these modifiable risk factors is to clearly quantify intraoperative risk of POD attributable to specific anesthetics. While preclinical studies have shown differential neurotoxicity effects of propofol and sevoflurane, their impact on clinically important neurologic outcomes such as delirium and cognitive decline remains poorly understood. Although Studies 1 and 2 both provided head-to-head comparisons of propofol and sevoflurane as risk factors for POD in high-operative-stress surgeries in older patients, the results were inconsistent. That being said, this small incremental increase in knowledge was not unexpected in the course of discovery around a clinically complex research question. Importantly, these studies provided evidence regarding the methodological approaches that could be taken to further this line of research.

The mediating factors of the differences on neurologic outcomes between anesthetic agents are likely pharmacological, biological, and methodological. Pharmacologically, the differences between target receptors, such as GABAA (propofol, etomidate) or NMDA (ketamine), could be a defining feature in the difference in incidence of POD. Additionally, secondary actions of anesthetic agents on glycine, nicotinic, and acetylcholine receptors could play a role as well. Biologically, genes such as CYP2E1, CYP2B6, CYP2C9, GSTP1, UGT1A9, SULT1A1, and NQO1 have all been identified as genetic factors in the metabolism of anesthetics, and variations in such genes could result in different responses to anesthetics.² Methodologically, routes of anesthetic administration (eg, inhalation vs intravenous), preexisting anatomical structures, or confounding medical conditions (eg, lower respiratory volume due to older age) may influence POD incidence, duration, or severity. Moreover, methodological differences between Studies 1 and 2, such as surgeries performed (spinal vs TKR/THR), patient populations (South Korean vs Chinese), and the diagnosis and monitoring of delirium (retrospective screening and diagnosis vs prospective CAM/CAM-S) may impact delirium outcomes. Thus, these factors should be considered in the design of future clinical trials undertaken to investigate the effects of anesthetics on POD.

Given the high prevalence of delirium and its associated adverse outcomes in the immediate postoperative period in older patients, further research is warranted to determine how anesthetics affect POD in order to optimize perioperative care and mitigate risks in this vulnerable population. Moreover, parallel investigations into how anesthetics differentially impact the development of transient or longer-term cognitive impairment after a surgical procedure (ie, postoperative cognitive dysfunction) in older adults are urgently needed in order to improve their cognitive health.

Practice Points

• Intravenous propofol and inhalational sevoflurane may be differentially associated with incidence, duration, and severity of POD in geriatric surgical patients.
• Further larger-scale studies are warranted to clarify the role of anesthetic choice in POD in order to optimize surgical outcomes in older patients.

–Jared Doan, BS, and Fred Ko, MD
Icahn School of Medicine at Mount Sinai

Study 1 Overview (Chang et al)

Objective: To assess the incidence of postoperative delirium (POD) following propofol- vs sevoflurane-based anesthesia in geriatric spine surgery patients.

Design: Retrospective, single-blinded observational study of propofol- and sevoflurane-based anesthesia cohorts.

Setting and participants: Patients eligible for this study were aged 65 years or older admitted to the SMG-SNU Boramae Medical Center (Seoul, South Korea). All patients underwent general anesthesia either via intravenous propofol or inhalational sevoflurane for spine surgery between January 2015 and December 2019. Patients were retrospectively identified via electronic medical records. Patient exclusion criteria included preoperative delirium, history of dementia, psychiatric disease, alcoholism, hepatic or renal dysfunction, postoperative mechanical ventilation dependence, other surgery within the recent 6 months, maintenance of intraoperative anesthesia with combined anesthetics, or incomplete medical record.

Main outcome measures: The primary outcome was the incidence of POD after administration of propofol- and sevoflurane-based anesthesia during hospitalization. Patients were screened for POD regularly by attending nurses using the Nursing Delirium Screening Scale (disorientation, inappropriate behavior, inappropriate communication, hallucination, and psychomotor retardation) during the entirety of the patient’s hospital stay; if 1 or more screening criteria were met, a psychiatrist was consulted for the proper diagnosis and management of delirium. A psychiatric diagnosis was required for a case to be counted toward the incidence of POD in this study. Secondary outcomes included postoperative 30-day complications (angina, myocardial infarction, transient ischemic attack/stroke, pneumonia, deep vein thrombosis, pulmonary embolism, acute kidney injury, or infection) and length of postoperative hospital stay.

Main results: POD occurred in 29 patients (10.3%) out of the total cohort of 281. POD was more common in the sevoflurane group than in the propofol group (15.7% vs 5.0%; P = .003). Using multivariable logistic regression, inhalational sevoflurane was associated with an increased risk of POD as compared to propofol-based anesthesia (odds ratio [OR], 4.120; 95% CI, 1.549-10.954; P = .005). There was no association between choice of anesthetic and postoperative 30-day complications or the length of postoperative hospital stay. Both older age (OR, 1.242; 95% CI, 1.130-1.366; P < .001) and higher pain score at postoperative day 1 (OR, 1.338; 95% CI, 1.056-1.696; P = .016) were associated with increased risk of POD.

Conclusion: Propofol-based anesthesia was associated with a lower incidence of and risk for POD than sevoflurane-based anesthesia in older patients undergoing spine surgery.

Study 2 Overview (Mei et al)

Objective: To determine the incidence and duration of POD in older patients after total knee/hip replacement (TKR/THR) under intravenous propofol or inhalational sevoflurane general anesthesia.

Design: Randomized clinical trial of propofol and sevoflurane groups.

Setting and participants: This study was conducted at the Shanghai Tenth People’s Hospital and involved 209 participants enrolled between June 2016 and November 2019. All participants were 60 years of age or older, scheduled for TKR/THR surgery under general anesthesia, American Society of Anesthesiologists (ASA) class I to III, and assessed to be of normal cognitive function preoperatively via a Mini-Mental State Examination. Participant exclusion criteria included preexisting delirium as assessed by the Confusion Assessment Method (CAM), prior diagnosed neurological diseases (eg, Parkinson’s disease), prior diagnosed mental disorders (eg, schizophrenia), or impaired vision or hearing that would influence cognitive assessments. All participants were randomly assigned to either sevoflurane or propofol anesthesia for their surgery via a computer-generated list. Of these, 103 received inhalational sevoflurane and 106 received intravenous propofol. All participants received standardized postoperative care.

Main outcome measures: All participants were interviewed by investigators, who were blinded to the anesthesia regimen, twice daily on postoperative days 1, 2, and 3 using CAM and a CAM-based scoring system (CAM-S) to assess delirium severity. The CAM encapsulated 4 criteria: acute onset and fluctuating course, agitation, disorganized thinking, and altered level of consciousness. To diagnose delirium, both the first and second criteria must be met, in addition to either the third or fourth criterion. The averages of the scores across the 3 postoperative days indicated delirium severity, while the incidence and duration of delirium was assessed by the presence of delirium as determined by CAM on any postoperative day.

Main results: All eligible participants (N = 209; mean [SD] age 71.2 [6.7] years; 29.2% male) were included in the final analysis. The incidence of POD was not statistically different between the propofol and sevoflurane groups (33.0% vs 23.3%; P = .119, Chi-square test). It was estimated that 316 participants in each arm of the study were needed to detect statistical differences. The number of days of POD per person were higher with propofol anesthesia as compared to sevoflurane (0.5 [0.8] vs 0.3 [0.5]; P =  .049, Student’s t-test).

Conclusion: This underpowered study showed a 9.7% difference in the incidence of POD between older adults who received propofol (33.0%) and sevoflurane (23.3%) after THR/TKR. Further studies with a larger sample size are needed to compare general anesthetics and their role in POD.

Commentary

Delirium is characterized by an acute state of confusion with fluctuating mental status, inattention, disorganized thinking, and altered level of consciousness. It is often caused by medications and/or their related adverse effects, infections, electrolyte imbalances, and other clinical etiologies. Delirium often manifests in post-surgical settings, disproportionately affecting older patients and leading to increased risk of morbidity, mortality, hospital length of stay, and health care costs.¹ Intraoperative risk factors for POD are determined by the degree of operative stress (eg, lower-risk surgeries put the patient at reduced risk for POD as compared to higher-risk surgeries) and are additive to preexisting patient-specific risk factors, such as older age and functional impairment.¹ Because operative stress is associated with risk for POD, limiting operative stress in controlled ways, such as through the choice of anesthetic agent administered, may be a pragmatic way to manage operative risks and optimize outcomes, especially when serving a surgically vulnerable population.

In Study 1, Chang et al sought to assess whether 2 commonly utilized general anesthetics, propofol and sevoflurane, in older patients undergoing spine surgery differentially affected the incidence of POD. In this retrospective, single-blinded observational study of 281 geriatric patients, the researchers found that sevoflurane was associated with a higher risk of POD as compared to propofol. However, these anesthetics were not associated with surgical outcomes such as postoperative 30-day complications or the length of postoperative hospital stay. While these findings added new knowledge to this field of research, several limitations should be kept in mind when interpreting this study’s results. For instance, the sample size was relatively small, with all cases selected from a single center utilizing a retrospective analysis. In addition, although a standardized nursing screening tool was used as a method for delirium detection, hypoactive delirium or less symptomatic delirium may have been missed, which in turn would lead to an underestimation of POD incidence. The latter is a common limitation in delirium research.

In Study 2, Mei et al similarly explored the effects of general anesthetics on POD in older surgical patients. Specifically, using a randomized clinical trial design, the investigators compared propofol with sevoflurane in older patients who underwent TKR/THR, and their roles in POD severity and duration. Although the incidence of POD was higher in those who received propofol compared to sevoflurane, this trial was underpowered and the results did not reach statistical significance. In addition, while the duration of POD was slightly longer in the propofol group compared to the sevoflurane group (0.5 vs 0.3 days), it was unclear if this finding was clinically significant. Similar to many research studies in POD, limitations of Study 2 included a small sample size of 209 patients, with all participants enrolled from a single center. On the other hand, this study illustrated the feasibility of a method that allowed reproducible prospective assessment of POD time course using CAM and CAM-S.

Applications for Clinical Practice and System Implementation

The delineation of risk factors that contribute to delirium after surgery in older patients is key to mitigating risks for POD and improving clinical outcomes. An important step towards a better understanding of these modifiable risk factors is to clearly quantify intraoperative risk of POD attributable to specific anesthetics. While preclinical studies have shown differential neurotoxicity effects of propofol and sevoflurane, their impact on clinically important neurologic outcomes such as delirium and cognitive decline remains poorly understood. Although Studies 1 and 2 both provided head-to-head comparisons of propofol and sevoflurane as risk factors for POD in high-operative-stress surgeries in older patients, the results were inconsistent. That being said, this small incremental increase in knowledge was not unexpected in the course of discovery around a clinically complex research question. Importantly, these studies provided evidence regarding the methodological approaches that could be taken to further this line of research.

The mediating factors of the differences on neurologic outcomes between anesthetic agents are likely pharmacological, biological, and methodological. Pharmacologically, the differences between target receptors, such as GABAA (propofol, etomidate) or NMDA (ketamine), could be a defining feature in the difference in incidence of POD. Additionally, secondary actions of anesthetic agents on glycine, nicotinic, and acetylcholine receptors could play a role as well. Biologically, genes such as CYP2E1, CYP2B6, CYP2C9, GSTP1, UGT1A9, SULT1A1, and NQO1 have all been identified as genetic factors in the metabolism of anesthetics, and variations in such genes could result in different responses to anesthetics.² Methodologically, routes of anesthetic administration (eg, inhalation vs intravenous), preexisting anatomical structures, or confounding medical conditions (eg, lower respiratory volume due to older age) may influence POD incidence, duration, or severity. Moreover, methodological differences between Studies 1 and 2, such as surgeries performed (spinal vs TKR/THR), patient populations (South Korean vs Chinese), and the diagnosis and monitoring of delirium (retrospective screening and diagnosis vs prospective CAM/CAM-S) may impact delirium outcomes. Thus, these factors should be considered in the design of future clinical trials undertaken to investigate the effects of anesthetics on POD.

Given the high prevalence of delirium and its associated adverse outcomes in the immediate postoperative period in older patients, further research is warranted to determine how anesthetics affect POD in order to optimize perioperative care and mitigate risks in this vulnerable population. Moreover, parallel investigations into how anesthetics differentially impact the development of transient or longer-term cognitive impairment after a surgical procedure (ie, postoperative cognitive dysfunction) in older adults are urgently needed in order to improve their cognitive health.

Practice Points

• Intravenous propofol and inhalational sevoflurane may be differentially associated with incidence, duration, and severity of POD in geriatric surgical patients.
• Further larger-scale studies are warranted to clarify the role of anesthetic choice in POD in order to optimize surgical outcomes in older patients.

–Jared Doan, BS, and Fred Ko, MD
Icahn School of Medicine at Mount Sinai

From the Department of Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC.

Abstract

Objective: Inpatient vaccination initiatives are well described in the literature. During the COVID-19 pandemic, hospitals began administering COVID-19 vaccines to hospitalized patients. Although vaccination rates increased, there remained many unvaccinated patients despite community efforts. This quality improvement project aimed to increase the COVID-19 vaccination rates of hospitalized patients on the medicine service at the George Washington University Hospital (GWUH).

Methods: From November 2021 through February 2022, we conducted a Plan-Do-Study-Act (PDSA) cycle with 3 phases. Initial steps included gathering baseline data from the electronic health record and consulting stakeholders. The first 2 phases focused on educating housestaff on the availability, ordering process, and administration of the Pfizer vaccine. The third phase consisted of developing educational pamphlets for patients to be included in their admission packets.

Results: The baseline mean COVID-19 vaccination rate (August to October 2021) of eligible patients on the medicine service was 10.7%. In the months after we implemented the PDSA cycle (November 2021 to February 2022), the mean vaccination rate increased to 15.4%.

Conclusion: This quality improvement project implemented measures to increase administration of the Pfizer vaccine to eligible patients admitted to the medicine service at GWUH. The mean vaccination rate increased from 10.7% in the 3 months prior to implementation to 15.4% during the 4 months post implementation. Other measures to consider in the future include increasing the availability of other COVID-19 vaccines at our hospital and incorporating the vaccine into the admission order set to help facilitate vaccination early in the hospital course.

Keywords: housestaff, quality improvement, PDSA, COVID-19, BNT162b2 vaccine, patient education

Throughout the COVID-19 pandemic, case rates in the United States have fluctuated considerably, corresponding to epidemic waves. In 2021, US daily cases of COVID-19 peaked at nearly 300,000 in early January and reached a nadir of 8000 cases in mid-June.¹ In September 2021, new cases had increased to 200,000 per day due to the prevalence of the Delta variant.¹ Particularly with the emergence of new variants of SARS-CoV-2, vaccination efforts to limit the spread of infection and severity of illness are critical. Data have shown that 2 doses of the BNT162b2 vaccine (Pfizer-BioNTech) were largely protective against severe infection for approximately 6 months.^2,3 When we began this quality improvement (QI) project in September 2021, only 179 million Americans had been fully vaccinated, according to data from the Centers for Disease Control and Prevention, which is just over half of the US population.⁴ An electronic survey conducted in the United States with more than 5 million responses found that, of those who were hesitant about receiving the vaccine, 49% reported a fear of adverse effects and 48% reported a lack of trust in the vaccine.⁵

This QI project sought to target unvaccinated individuals admitted to the internal medicine inpatient service. Vaccinating hospitalized patients is especially important since they are sicker than the general population and at higher risk of having poor outcomes from COVID-19. Inpatient vaccine initiatives, such as administering influenza vaccine prior to discharge, have been successfully implemented in the past.⁶ One large COVID-19 vaccination program featured an admission order set to increase the rates of vaccination among hospitalized patients.⁷ Our QI project piloted a multidisciplinary approach involving the nursing staff, pharmacy, information technology (IT) department, and internal medicine housestaff to increase COVID-19 vaccination rates among hospitalized patients on the medical service. This project aimed to increase inpatient vaccination rates through interventions targeting both primary providers as well as the patients themselves.

Methods

Setting and Interventions

This project was conducted at the George Washington University Hospital (GWUH) in Washington, DC. The clinicians involved in the study were the internal medicine housestaff, and the patients included were adults admitted to the resident medicine ward teams. The project was exempt by the institutional review board and did not require informed consent.

The quality improvement initiative had 3 phases, each featuring a different intervention (Table 1). The first phase involved sending a weekly announcement (via email and a secure health care messaging app) to current residents rotating on the inpatient medicine service. The announcement contained information regarding COVID-19 vaccine availability at the hospital, instructions on ordering the vaccine, and the process of coordinating with pharmacy to facilitate vaccine administration. Thereafter, residents were educated on the process of giving a COVID-19 vaccine to a patient from start to finish. Due to the nature of the residency schedule, different housestaff members rotated in and out of the medicine wards during the intervention periods. The weekly email was sent to the entire internal medicine housestaff, informing all residents about the QI project, while the weekly secure messages served as reminders and were only sent to residents currently on the medicine wards.

In the second phase, we posted paper flyers throughout the hospital to remind housestaff to give the vaccine and again educate them on the process of ordering the vaccine. For the third intervention, a COVID-19 vaccine educational pamphlet was developed for distribution to inpatients at GWUH. The pamphlet included information on vaccine efficacy, safety, side effects, and eligibility. The pamphlet was incorporated in the admission packet that every patient receives upon admission to the hospital. The patients reviewed the pamphlets with nursing staff, who would answer any questions, with residents available to discuss any outstanding concerns.

Measures and Data Gathering

The primary endpoint of the study was inpatient vaccination rate, defined as the number of COVID-19 vaccines administered divided by the number of patients eligible to receive a vaccine (not fully vaccinated). During initial triage, nursing staff documented vaccination status in the electronic health record (EHR), checking a box in a data entry form if a patient had received 0, 1, or 2 doses of the COVID-19 vaccine. The GWUH IT department generated data from this form to determine the number of patients eligible to receive a COVID-19 vaccine. Data were extracted from the medication administration record in the EHR to determine the number of vaccines that were administered to patients during their hospitalization on the inpatient medical service. Each month, the IT department extracted data for the number of eligible patients and the number of vaccines administered. This yielded the monthly vaccination rates. The monthly vaccination rates in the period prior to starting the QI initiative were compared to the rates in the period after the interventions were implemented.

Of note, during the course of this project, patients became eligible for a third COVID-19 vaccine (booster). We decided to continue with the original aim of vaccinating adults who had only received 0 or 1 dose of the vaccine. Therefore, the eligibility criteria remained the same throughout the study. We obtained retrospective data to ensure that the vaccines being counted toward the vaccination rate were vaccines given to patients not yet fully vaccinated and not vaccines given as boosters.

Results

From August to October 2021, the baseline average monthly vaccination rate of patients on the medicine service who were eligible to receive a COVID-19 vaccine was 10.7%. After the first intervention, the vaccination rate increased to 19.7% in November 2021 (Table 2). The second intervention yielded vaccination rates of 11.4% and 11.8% in December 2021 and January 2022, respectively. During the final phase in February 2022, the vaccination rate was 19.0%. At the conclusion of the study, the mean vaccination rate for the intervention months was 15.4% (Figure 1). Process stability and variation are demonstrated with a statistical process control chart (Figure 2).

For this housestaff-driven QI project, we implemented an inpatient COVID-19 vaccination campaign consisting of 3 phases that targeted both providers and patients. During the intervention period, we observed an increased vaccination rate compared to the period just prior to implementation of the QI project. While our interventions may certainly have boosted vaccination rates, we understand other variables could have contributed to increased rates as well. The emergence of variants in the United States, such as omicron in December 2021,⁸ could have precipitated a demand for vaccinations among patients. Holidays in November and December may also have increased patients’ desire to get vaccinated before travel.

We encountered a number of roadblocks that challenged our project, including difficulty identifying patients who were eligible for the vaccine, logistical vaccine administration challenges, and hesitancy among the inpatient population. Accurately identifying patients who were eligible for a vaccine in the EHR was especially challenging in the setting of rapidly changing guidelines regarding COVID-19 vaccination. In September 2021, the US Food and Drug Administration authorized the Pfizer booster for certain populations and later, in November 2021, for all adults. This meant that some fully vaccinated hospitalized patients (those with 2 doses) then qualified for an additional dose of the vaccine and received a dose during hospitalization. To determine the true vaccination rate, we obtained retrospective data that allowed us to track each vaccine administered. If a patient had already received 2 doses of the COVID-19 vaccine, the vaccine administered was counted as a booster and excluded from the calculation of the vaccination rate. Future PDSA cycles could include updating the EHR to capture the whole range of COVID-19 vaccination status (unvaccinated, partially vaccinated, fully vaccinated, fully vaccinated with 1 booster, fully vaccinated with 2 boosters).

We also encountered logistical challenges with the administration of the COVID-19 vaccine to hospitalized patients. During the intervention period, our pharmacy department required 5 COVID-19 vaccination orders before opening a vial and administering the vaccine doses in order to reduce waste. This policy may have limited our ability to vaccinate eligible inpatients because we were not always able to identify 5 patients simultaneously on the service who were eligible and consented to the vaccine.

The majority of patients who were interested in receiving COVID-19 vaccination had already been vaccinated in the outpatient setting. This fact made the inpatient internal medicine subset of patients a particularly challenging population to target, given their possible hesitancy regarding vaccination. By utilizing a multidisciplinary team and increasing communication of providers and nursing staff, we helped to increase the COVID-19 vaccination rates at our hospital from 10.7% to 15.4%.

Future Directions

Future interventions to consider include increasing the availability of other approved COVID-19 vaccines at our hospital besides the Pfizer-BioNTech vaccine. Furthermore, incorporating the vaccine into the admission order set would help initiate the vaccination process early in the hospital course. We encourage other institutions to utilize similar approaches to not only remind providers about inpatient vaccination, but also educate and encourage patients to receive the vaccine. These measures will help institutions increase inpatient COVID-19 vaccination rates in a high-risk population.

Corresponding author: Anna Rubin, MD, Department of Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC; arubin@mfa.gwu.edu

Disclosures: None reported.

From the Department of Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC.

Abstract

Objective: Inpatient vaccination initiatives are well described in the literature. During the COVID-19 pandemic, hospitals began administering COVID-19 vaccines to hospitalized patients. Although vaccination rates increased, there remained many unvaccinated patients despite community efforts. This quality improvement project aimed to increase the COVID-19 vaccination rates of hospitalized patients on the medicine service at the George Washington University Hospital (GWUH).

Methods: From November 2021 through February 2022, we conducted a Plan-Do-Study-Act (PDSA) cycle with 3 phases. Initial steps included gathering baseline data from the electronic health record and consulting stakeholders. The first 2 phases focused on educating housestaff on the availability, ordering process, and administration of the Pfizer vaccine. The third phase consisted of developing educational pamphlets for patients to be included in their admission packets.

Results: The baseline mean COVID-19 vaccination rate (August to October 2021) of eligible patients on the medicine service was 10.7%. In the months after we implemented the PDSA cycle (November 2021 to February 2022), the mean vaccination rate increased to 15.4%.

Conclusion: This quality improvement project implemented measures to increase administration of the Pfizer vaccine to eligible patients admitted to the medicine service at GWUH. The mean vaccination rate increased from 10.7% in the 3 months prior to implementation to 15.4% during the 4 months post implementation. Other measures to consider in the future include increasing the availability of other COVID-19 vaccines at our hospital and incorporating the vaccine into the admission order set to help facilitate vaccination early in the hospital course.

Keywords: housestaff, quality improvement, PDSA, COVID-19, BNT162b2 vaccine, patient education

Throughout the COVID-19 pandemic, case rates in the United States have fluctuated considerably, corresponding to epidemic waves. In 2021, US daily cases of COVID-19 peaked at nearly 300,000 in early January and reached a nadir of 8000 cases in mid-June.¹ In September 2021, new cases had increased to 200,000 per day due to the prevalence of the Delta variant.¹ Particularly with the emergence of new variants of SARS-CoV-2, vaccination efforts to limit the spread of infection and severity of illness are critical. Data have shown that 2 doses of the BNT162b2 vaccine (Pfizer-BioNTech) were largely protective against severe infection for approximately 6 months.^2,3 When we began this quality improvement (QI) project in September 2021, only 179 million Americans had been fully vaccinated, according to data from the Centers for Disease Control and Prevention, which is just over half of the US population.⁴ An electronic survey conducted in the United States with more than 5 million responses found that, of those who were hesitant about receiving the vaccine, 49% reported a fear of adverse effects and 48% reported a lack of trust in the vaccine.⁵

This QI project sought to target unvaccinated individuals admitted to the internal medicine inpatient service. Vaccinating hospitalized patients is especially important since they are sicker than the general population and at higher risk of having poor outcomes from COVID-19. Inpatient vaccine initiatives, such as administering influenza vaccine prior to discharge, have been successfully implemented in the past.⁶ One large COVID-19 vaccination program featured an admission order set to increase the rates of vaccination among hospitalized patients.⁷ Our QI project piloted a multidisciplinary approach involving the nursing staff, pharmacy, information technology (IT) department, and internal medicine housestaff to increase COVID-19 vaccination rates among hospitalized patients on the medical service. This project aimed to increase inpatient vaccination rates through interventions targeting both primary providers as well as the patients themselves.

Methods

Setting and Interventions

This project was conducted at the George Washington University Hospital (GWUH) in Washington, DC. The clinicians involved in the study were the internal medicine housestaff, and the patients included were adults admitted to the resident medicine ward teams. The project was exempt by the institutional review board and did not require informed consent.

The quality improvement initiative had 3 phases, each featuring a different intervention (Table 1). The first phase involved sending a weekly announcement (via email and a secure health care messaging app) to current residents rotating on the inpatient medicine service. The announcement contained information regarding COVID-19 vaccine availability at the hospital, instructions on ordering the vaccine, and the process of coordinating with pharmacy to facilitate vaccine administration. Thereafter, residents were educated on the process of giving a COVID-19 vaccine to a patient from start to finish. Due to the nature of the residency schedule, different housestaff members rotated in and out of the medicine wards during the intervention periods. The weekly email was sent to the entire internal medicine housestaff, informing all residents about the QI project, while the weekly secure messages served as reminders and were only sent to residents currently on the medicine wards.

In the second phase, we posted paper flyers throughout the hospital to remind housestaff to give the vaccine and again educate them on the process of ordering the vaccine. For the third intervention, a COVID-19 vaccine educational pamphlet was developed for distribution to inpatients at GWUH. The pamphlet included information on vaccine efficacy, safety, side effects, and eligibility. The pamphlet was incorporated in the admission packet that every patient receives upon admission to the hospital. The patients reviewed the pamphlets with nursing staff, who would answer any questions, with residents available to discuss any outstanding concerns.

Measures and Data Gathering

The primary endpoint of the study was inpatient vaccination rate, defined as the number of COVID-19 vaccines administered divided by the number of patients eligible to receive a vaccine (not fully vaccinated). During initial triage, nursing staff documented vaccination status in the electronic health record (EHR), checking a box in a data entry form if a patient had received 0, 1, or 2 doses of the COVID-19 vaccine. The GWUH IT department generated data from this form to determine the number of patients eligible to receive a COVID-19 vaccine. Data were extracted from the medication administration record in the EHR to determine the number of vaccines that were administered to patients during their hospitalization on the inpatient medical service. Each month, the IT department extracted data for the number of eligible patients and the number of vaccines administered. This yielded the monthly vaccination rates. The monthly vaccination rates in the period prior to starting the QI initiative were compared to the rates in the period after the interventions were implemented.

Of note, during the course of this project, patients became eligible for a third COVID-19 vaccine (booster). We decided to continue with the original aim of vaccinating adults who had only received 0 or 1 dose of the vaccine. Therefore, the eligibility criteria remained the same throughout the study. We obtained retrospective data to ensure that the vaccines being counted toward the vaccination rate were vaccines given to patients not yet fully vaccinated and not vaccines given as boosters.

Results

From August to October 2021, the baseline average monthly vaccination rate of patients on the medicine service who were eligible to receive a COVID-19 vaccine was 10.7%. After the first intervention, the vaccination rate increased to 19.7% in November 2021 (Table 2). The second intervention yielded vaccination rates of 11.4% and 11.8% in December 2021 and January 2022, respectively. During the final phase in February 2022, the vaccination rate was 19.0%. At the conclusion of the study, the mean vaccination rate for the intervention months was 15.4% (Figure 1). Process stability and variation are demonstrated with a statistical process control chart (Figure 2).

For this housestaff-driven QI project, we implemented an inpatient COVID-19 vaccination campaign consisting of 3 phases that targeted both providers and patients. During the intervention period, we observed an increased vaccination rate compared to the period just prior to implementation of the QI project. While our interventions may certainly have boosted vaccination rates, we understand other variables could have contributed to increased rates as well. The emergence of variants in the United States, such as omicron in December 2021,⁸ could have precipitated a demand for vaccinations among patients. Holidays in November and December may also have increased patients’ desire to get vaccinated before travel.

We encountered a number of roadblocks that challenged our project, including difficulty identifying patients who were eligible for the vaccine, logistical vaccine administration challenges, and hesitancy among the inpatient population. Accurately identifying patients who were eligible for a vaccine in the EHR was especially challenging in the setting of rapidly changing guidelines regarding COVID-19 vaccination. In September 2021, the US Food and Drug Administration authorized the Pfizer booster for certain populations and later, in November 2021, for all adults. This meant that some fully vaccinated hospitalized patients (those with 2 doses) then qualified for an additional dose of the vaccine and received a dose during hospitalization. To determine the true vaccination rate, we obtained retrospective data that allowed us to track each vaccine administered. If a patient had already received 2 doses of the COVID-19 vaccine, the vaccine administered was counted as a booster and excluded from the calculation of the vaccination rate. Future PDSA cycles could include updating the EHR to capture the whole range of COVID-19 vaccination status (unvaccinated, partially vaccinated, fully vaccinated, fully vaccinated with 1 booster, fully vaccinated with 2 boosters).

We also encountered logistical challenges with the administration of the COVID-19 vaccine to hospitalized patients. During the intervention period, our pharmacy department required 5 COVID-19 vaccination orders before opening a vial and administering the vaccine doses in order to reduce waste. This policy may have limited our ability to vaccinate eligible inpatients because we were not always able to identify 5 patients simultaneously on the service who were eligible and consented to the vaccine.

The majority of patients who were interested in receiving COVID-19 vaccination had already been vaccinated in the outpatient setting. This fact made the inpatient internal medicine subset of patients a particularly challenging population to target, given their possible hesitancy regarding vaccination. By utilizing a multidisciplinary team and increasing communication of providers and nursing staff, we helped to increase the COVID-19 vaccination rates at our hospital from 10.7% to 15.4%.

Future Directions

Future interventions to consider include increasing the availability of other approved COVID-19 vaccines at our hospital besides the Pfizer-BioNTech vaccine. Furthermore, incorporating the vaccine into the admission order set would help initiate the vaccination process early in the hospital course. We encourage other institutions to utilize similar approaches to not only remind providers about inpatient vaccination, but also educate and encourage patients to receive the vaccine. These measures will help institutions increase inpatient COVID-19 vaccination rates in a high-risk population.

Corresponding author: Anna Rubin, MD, Department of Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC; arubin@mfa.gwu.edu

Disclosures: None reported.

From the Department of Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC.

Abstract

Objective: Inpatient vaccination initiatives are well described in the literature. During the COVID-19 pandemic, hospitals began administering COVID-19 vaccines to hospitalized patients. Although vaccination rates increased, there remained many unvaccinated patients despite community efforts. This quality improvement project aimed to increase the COVID-19 vaccination rates of hospitalized patients on the medicine service at the George Washington University Hospital (GWUH).

Methods: From November 2021 through February 2022, we conducted a Plan-Do-Study-Act (PDSA) cycle with 3 phases. Initial steps included gathering baseline data from the electronic health record and consulting stakeholders. The first 2 phases focused on educating housestaff on the availability, ordering process, and administration of the Pfizer vaccine. The third phase consisted of developing educational pamphlets for patients to be included in their admission packets.

Results: The baseline mean COVID-19 vaccination rate (August to October 2021) of eligible patients on the medicine service was 10.7%. In the months after we implemented the PDSA cycle (November 2021 to February 2022), the mean vaccination rate increased to 15.4%.

Conclusion: This quality improvement project implemented measures to increase administration of the Pfizer vaccine to eligible patients admitted to the medicine service at GWUH. The mean vaccination rate increased from 10.7% in the 3 months prior to implementation to 15.4% during the 4 months post implementation. Other measures to consider in the future include increasing the availability of other COVID-19 vaccines at our hospital and incorporating the vaccine into the admission order set to help facilitate vaccination early in the hospital course.

Keywords: housestaff, quality improvement, PDSA, COVID-19, BNT162b2 vaccine, patient education

Throughout the COVID-19 pandemic, case rates in the United States have fluctuated considerably, corresponding to epidemic waves. In 2021, US daily cases of COVID-19 peaked at nearly 300,000 in early January and reached a nadir of 8000 cases in mid-June.¹ In September 2021, new cases had increased to 200,000 per day due to the prevalence of the Delta variant.¹ Particularly with the emergence of new variants of SARS-CoV-2, vaccination efforts to limit the spread of infection and severity of illness are critical. Data have shown that 2 doses of the BNT162b2 vaccine (Pfizer-BioNTech) were largely protective against severe infection for approximately 6 months.^2,3 When we began this quality improvement (QI) project in September 2021, only 179 million Americans had been fully vaccinated, according to data from the Centers for Disease Control and Prevention, which is just over half of the US population.⁴ An electronic survey conducted in the United States with more than 5 million responses found that, of those who were hesitant about receiving the vaccine, 49% reported a fear of adverse effects and 48% reported a lack of trust in the vaccine.⁵

This QI project sought to target unvaccinated individuals admitted to the internal medicine inpatient service. Vaccinating hospitalized patients is especially important since they are sicker than the general population and at higher risk of having poor outcomes from COVID-19. Inpatient vaccine initiatives, such as administering influenza vaccine prior to discharge, have been successfully implemented in the past.⁶ One large COVID-19 vaccination program featured an admission order set to increase the rates of vaccination among hospitalized patients.⁷ Our QI project piloted a multidisciplinary approach involving the nursing staff, pharmacy, information technology (IT) department, and internal medicine housestaff to increase COVID-19 vaccination rates among hospitalized patients on the medical service. This project aimed to increase inpatient vaccination rates through interventions targeting both primary providers as well as the patients themselves.

Methods

Setting and Interventions

This project was conducted at the George Washington University Hospital (GWUH) in Washington, DC. The clinicians involved in the study were the internal medicine housestaff, and the patients included were adults admitted to the resident medicine ward teams. The project was exempt by the institutional review board and did not require informed consent.

The quality improvement initiative had 3 phases, each featuring a different intervention (Table 1). The first phase involved sending a weekly announcement (via email and a secure health care messaging app) to current residents rotating on the inpatient medicine service. The announcement contained information regarding COVID-19 vaccine availability at the hospital, instructions on ordering the vaccine, and the process of coordinating with pharmacy to facilitate vaccine administration. Thereafter, residents were educated on the process of giving a COVID-19 vaccine to a patient from start to finish. Due to the nature of the residency schedule, different housestaff members rotated in and out of the medicine wards during the intervention periods. The weekly email was sent to the entire internal medicine housestaff, informing all residents about the QI project, while the weekly secure messages served as reminders and were only sent to residents currently on the medicine wards.

In the second phase, we posted paper flyers throughout the hospital to remind housestaff to give the vaccine and again educate them on the process of ordering the vaccine. For the third intervention, a COVID-19 vaccine educational pamphlet was developed for distribution to inpatients at GWUH. The pamphlet included information on vaccine efficacy, safety, side effects, and eligibility. The pamphlet was incorporated in the admission packet that every patient receives upon admission to the hospital. The patients reviewed the pamphlets with nursing staff, who would answer any questions, with residents available to discuss any outstanding concerns.

Measures and Data Gathering

The primary endpoint of the study was inpatient vaccination rate, defined as the number of COVID-19 vaccines administered divided by the number of patients eligible to receive a vaccine (not fully vaccinated). During initial triage, nursing staff documented vaccination status in the electronic health record (EHR), checking a box in a data entry form if a patient had received 0, 1, or 2 doses of the COVID-19 vaccine. The GWUH IT department generated data from this form to determine the number of patients eligible to receive a COVID-19 vaccine. Data were extracted from the medication administration record in the EHR to determine the number of vaccines that were administered to patients during their hospitalization on the inpatient medical service. Each month, the IT department extracted data for the number of eligible patients and the number of vaccines administered. This yielded the monthly vaccination rates. The monthly vaccination rates in the period prior to starting the QI initiative were compared to the rates in the period after the interventions were implemented.

Of note, during the course of this project, patients became eligible for a third COVID-19 vaccine (booster). We decided to continue with the original aim of vaccinating adults who had only received 0 or 1 dose of the vaccine. Therefore, the eligibility criteria remained the same throughout the study. We obtained retrospective data to ensure that the vaccines being counted toward the vaccination rate were vaccines given to patients not yet fully vaccinated and not vaccines given as boosters.

Results

From August to October 2021, the baseline average monthly vaccination rate of patients on the medicine service who were eligible to receive a COVID-19 vaccine was 10.7%. After the first intervention, the vaccination rate increased to 19.7% in November 2021 (Table 2). The second intervention yielded vaccination rates of 11.4% and 11.8% in December 2021 and January 2022, respectively. During the final phase in February 2022, the vaccination rate was 19.0%. At the conclusion of the study, the mean vaccination rate for the intervention months was 15.4% (Figure 1). Process stability and variation are demonstrated with a statistical process control chart (Figure 2).

For this housestaff-driven QI project, we implemented an inpatient COVID-19 vaccination campaign consisting of 3 phases that targeted both providers and patients. During the intervention period, we observed an increased vaccination rate compared to the period just prior to implementation of the QI project. While our interventions may certainly have boosted vaccination rates, we understand other variables could have contributed to increased rates as well. The emergence of variants in the United States, such as omicron in December 2021,⁸ could have precipitated a demand for vaccinations among patients. Holidays in November and December may also have increased patients’ desire to get vaccinated before travel.

We encountered a number of roadblocks that challenged our project, including difficulty identifying patients who were eligible for the vaccine, logistical vaccine administration challenges, and hesitancy among the inpatient population. Accurately identifying patients who were eligible for a vaccine in the EHR was especially challenging in the setting of rapidly changing guidelines regarding COVID-19 vaccination. In September 2021, the US Food and Drug Administration authorized the Pfizer booster for certain populations and later, in November 2021, for all adults. This meant that some fully vaccinated hospitalized patients (those with 2 doses) then qualified for an additional dose of the vaccine and received a dose during hospitalization. To determine the true vaccination rate, we obtained retrospective data that allowed us to track each vaccine administered. If a patient had already received 2 doses of the COVID-19 vaccine, the vaccine administered was counted as a booster and excluded from the calculation of the vaccination rate. Future PDSA cycles could include updating the EHR to capture the whole range of COVID-19 vaccination status (unvaccinated, partially vaccinated, fully vaccinated, fully vaccinated with 1 booster, fully vaccinated with 2 boosters).

We also encountered logistical challenges with the administration of the COVID-19 vaccine to hospitalized patients. During the intervention period, our pharmacy department required 5 COVID-19 vaccination orders before opening a vial and administering the vaccine doses in order to reduce waste. This policy may have limited our ability to vaccinate eligible inpatients because we were not always able to identify 5 patients simultaneously on the service who were eligible and consented to the vaccine.

The majority of patients who were interested in receiving COVID-19 vaccination had already been vaccinated in the outpatient setting. This fact made the inpatient internal medicine subset of patients a particularly challenging population to target, given their possible hesitancy regarding vaccination. By utilizing a multidisciplinary team and increasing communication of providers and nursing staff, we helped to increase the COVID-19 vaccination rates at our hospital from 10.7% to 15.4%.

Future Directions

Future interventions to consider include increasing the availability of other approved COVID-19 vaccines at our hospital besides the Pfizer-BioNTech vaccine. Furthermore, incorporating the vaccine into the admission order set would help initiate the vaccination process early in the hospital course. We encourage other institutions to utilize similar approaches to not only remind providers about inpatient vaccination, but also educate and encourage patients to receive the vaccine. These measures will help institutions increase inpatient COVID-19 vaccination rates in a high-risk population.

Corresponding author: Anna Rubin, MD, Department of Medicine, The George Washington University School of Medicine and Health Sciences, Washington, DC; arubin@mfa.gwu.edu

Disclosures: None reported.

From the T1D Exchange, Boston, MA (Ann Mungmode, Nicole Rioles, Jesse Cases, Dr. Ebekozien); The Leona M. and Harry B. Hemsley Charitable Trust, New York, NY (Laurel Koester); and the University of Mississippi School of Population Health, Jackson, MS (Dr. Ebekozien).

Abstract

There have been remarkable innovations in diabetes management since the start of the COVID-19 pandemic, but these groundbreaking innovations are drawing limited focus as the field focuses on the adverse impact of the pandemic on patients with diabetes. This article reviews select population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of the T1D Exchange Quality Improvement Collaborative, a learning health network that focuses on improving care and outcomes for individuals with type 1 diabetes (T1D). Such innovations include expanded telemedicine access, collection of real-world data, machine learning and artificial intelligence, and new diabetes medications and devices. In addition, multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and advocacy efforts for specific populations have been successful. Looking to the future, work is required to explore additional health equity successes that do not further exacerbate inequities and to look for additional innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Keywords: type 1 diabetes, learning health network, continuous glucose monitoring, health equity

One in 10 people in the United States has diabetes.¹ Diabetes is the nation’s second leading cause of death, costing the US health system more than $300 billion annually.² The COVID-19 pandemic presented additional health burdens for people living with diabetes. For example, preexisting diabetes was identified as a risk factor for COVID-19–associated morbidity and mortality.^3,4 Over the past 2 years, there have been remarkable innovations in diabetes management, including stem cell therapy and new medication options. Additionally, improved technology solutions have aided in diabetes management through continuous glucose monitors (CGM), smart insulin pens, advanced hybrid closed-loop systems, and continuous subcutaneous insulin injections.^5,6 Unfortunately, these groundbreaking innovations are drawing limited focus, as the field is rightfully focused on the adverse impact of the pandemic on patients with diabetes.

Learning health networks like the T1D Exchange Quality Improvement Collaborative (T1DX-QI) have implemented some of these innovative solutions to improve care for people with diabetes.⁷ T1DX-QI has more than 50 data-sharing endocrinology centers that care for over 75,000 people with diabetes across the United States (Figure 1). Centers participating in the T1DX-QI use quality improvement (QI) and implementation science methods to quickly translate research into evidence-based clinical practice. T1DX-QI leads diabetes population health and health system research and supports widespread transferability across health care organizations through regular collaborative calls, conferences, and case study documentation.⁸

In this review, we summarize impactful population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of T1DX-QI (see Figure 2 for relevant definitions). This review is limited in scope and is not meant to be an exhaustive list of innovations. The review also reflects significant changes from the perspective of academic diabetes centers, which may not apply to rural or primary care diabetes practices.

The first (A.M.), second (H.H.), and senior (O.E.) authors conducted a scoping review of published literature using terms related to diabetes, population health, and innovation on PubMed Central and Google Scholar for the period March 2020 to June 2022. To complement the review, A.M. and O.E. also reviewed abstracts from presentations at major international diabetes conferences, including the American Diabetes Association (ADA), the International Society for Pediatric and Adolescent Diabetes (ISPAD), the T1DX-QI Learning Session Conference, and the Advanced Technologies & Treatments for Diabetes (ATTD) 2020 to 2022 conferences.^9-14 The authors also searched FDA.gov and ClinicalTrials.gov for relevant insights. A.M. and O.E. sorted the reviewed literature into major themes (Figure 3) from the population health improvement perspective of the T1DX-QI.

Population Health Innovations in Diabetes Management

Expansion of Telemedicine Access

Telemedicine is cost-effective for patients with diabetes,¹⁵ including those with complex cases.¹⁶ Before the COVID-19 pandemic, telemedicine and virtual care were rare in diabetes management. However, the pandemic offered a new opportunity to expand the practice of telemedicine in diabetes management. A study from the T1DX-QI showed that telemedicine visits grew from comprising <1% of visits pre-pandemic (December 2019) to 95.2% during the pandemic (August 2020).¹⁷ Additional studies, like those conducted by Phillip et al,¹⁸ confirmed the noninferiority of telemedicine practice for patients with diabetes.Telemedicine was also found to be an effective strategy to educate patients on the use of diabetes technologies.¹⁹

Real-World Data and Disease Surveillance

As the COVID-19 pandemic exacerbated outcomes for people with type 1 diabetes (T1D), a need arose to understand the immediate effects of the pandemic on people with T1D through real-world data and disease surveillance. In April 2020, the T1DX-QI initiated a multicenter surveillance study to collect data and analyze the impact of COVID-19 on people with T1D. The existing health collaborative served as a springboard for robust surveillance study, documenting numerous works on the effects of COVID-19.^3,4,20-28 Other investigators also embraced the power of real-world surveillance and real-world data.^29,30

Big Data, Machine Learning, and Artificial Intelligence

The past 2 years have seen a shift toward embracing the incredible opportunity to tap the large volume of data generated from routine care for practical insights.³¹ In particular, researchers have demonstrated the widespread application of machine learning and artificial intelligence to improve diabetes management.³² The T1DX-QI also harnessed the growing power of big data by expanding the functionality of innovative benchmarking software. The T1DX QI Portal uses electronic medical record data of diabetes patients for clinic-to-clinic benchmarking and data analysis, using business intelligence solutions.³³

Health Equity

While inequities across various health outcomes have been well documented for years,³⁴ the COVID-19 pandemic further exaggerated racial/ethnic health inequities in T1D.^23,35 In response, several organizations have outlined specific strategies to address these health inequities. Emboldened by the pandemic, the T1DX-QI announced a multipronged approach to address health inequities among patients with T1D through the Health Equity Advancement Lab (HEAL).³⁶ One of HEAL’s main components is using real-world data to champion population-level insights and demonstrate progress in QI efforts.

Multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and these studies are expanding our understanding of the chasm.³⁷ There have also been innovative solutions to addressing these inequities, with multiple studies published over the past 2 years.³⁸ A source of inequity among patients with T1D is the lack of representation of racial/ethnic minorities with T1D in clinical trials.³⁹ The T1DX-QI suggests that the equity-adapted framework for QI can be applied by research leaders to support trial diversity and representation, ensuring future device innovations are meaningful for all people with T1D.⁴⁰

Diabetes Devices

Glucose monitoring and insulin therapy are vital tools to support individuals living with T1D, and devices such as CGM and insulin pumps have become the standard of care for diabetes management (Table).⁴¹ Innovations in diabetes technology and device access are imperative for a chronic disease with no cure.

The COVID-19 pandemic created an opportunity to increase access to diabetes devices in inpatient settings. In 2020, the US Food and Drug Administration expanded the use of CGM to support remote monitoring of patients in inpatient hospital settings, simultaneously supporting the glucose monitoring needs of patients with T1D and reducing COVID-19 transmission through reduced patient-clinician contact.⁴² This effort has been expanded and will continue in 2022 and beyond,⁴³ and aligns with the growing consensus that supports patients wearing both CGMs and insulin pumps in ambulatory settings to improve patient health outcomes.⁴⁴

Since 2020, innovations in diabetes technology have improved and increased the variety of options available to people with T1D and made them easier to use (Table). New, advanced hybrid closed-loop systems have progressed to offer Bluetooth features, including automatic software upgrades, tubeless systems, and the ability to allow parents to use their smartphones to bolus for children.^45-47 The next big step in insulin delivery innovation is the release of functioning, fully closed loop systems, of which several are currently in clinical trials.⁴⁸ These systems support reduced hypoglycemia and improved time in range.⁴⁹

Additional innovations in insulin delivery have improved the user experience and expanded therapeutic options, including a variety of smart insulin pens complete with dosing logs^50,51 and even a patch to deliver insulin without the burden of injections.⁵² As barriers to diabetes technology persist,⁵³ innovations in alternate insulin delivery provide people with T1D more options to align with their personal access and technology preferences.

Innovations in CGM address cited barriers to their use, including size or overall wear.^53-55 CGMs released in the past few years are smaller in physical size, have longer durations of time between changings, are more accurate, and do not require calibrations for accuracy.

New Diabetes Medications

Many new medications and therapeutic advances have become available in the past 2 years.⁵⁶ Additionally, more medications are being tested as adjunct therapies to support glycemic management in patients with T1D, including metformin, sodium-glucose cotransporter 1 and 2 inhibitors, pramlintide, glucagon-like polypeptide-1 analogs, and glucagon receptor agonists.⁵⁷ Other recent advances include stem cell replacement therapy for patients with T1D.⁵⁸ The ultra-long-acting biosimilar insulins are one medical innovation that has been stalled, rather than propelled, during the COVID-19 pandemic.⁵⁹

Diabetes Policy Advocacy

People with T1D require insulin to survive. The cost of insulin has increased in recent years, with some studies citing a 64% to 100% increase in the past decade.^60,61 In fact, 1 in 4 insulin users report that cost has impacted their insulin use, including rationing their insulin.⁶² Lockdowns during the COVID-19 pandemic stressed US families financially, increasing the urgency for insulin cost caps.

Although the COVID-19 pandemic halted national conversations on drug financing,⁶³ advocacy efforts have succeeded for specific populations. The new Medicare Part D Senior Savings Model will cap the cost of insulin at $35 for a 30-day supply,⁶⁴ and 20 states passed legislation capping insulin pricing.⁶² Efforts to codify national cost caps are under debate, including the passage of the Affordable Insulin Now Act, which passed the House in March 2022 and is currently under review in the Senate.⁶⁵

Funders and industry partners play a crucial role in leading and supporting innovations that improve the lives of people with T1D and reduce society’s costs of living with the disease. Data infrastructure is critical to supporting population health. While building the data infrastructure to support population health is both time- and resource-intensive, private foundations such as Helmsley are uniquely positioned—and have a responsibility—to take large, informed risks to help reach all communities with T1D.

The T1DX-QI is the largest source of population health data on T1D in the United States and is becoming the premiere data authority on its incidence, prevalence, and outcomes. The T1DX-QI enables a robust understanding of T1D-related health trends at the population level, as well as trends among clinics and providers. Pilot centers in the T1DX-QI have reported reductions in patients’ A1c and acute diabetes-related events, as well as improvements in device usage and depression screening. The ability to capture changes speaks to the promise and power of these data to demonstrate the clinical impact of QI interventions and to support the spread of best practices and learnings across health systems.

Additional philanthropic efforts have supported innovation in the last 2 years. For example, the JDRF, a nonprofit philanthropic equity firm, has supported efforts in developing artificial pancreas systems and cell therapies currently in clinical trials like teplizumab, a drug that has demonstrated delayed onset of T1D through JDRF’s T1D Fund.⁶⁶ Industry partners also have an opportunity for significant influence in this area, as they continue to fund meaningful projects to advance care for people with T1D.⁶⁷

Conclusion

We are optimistic that the innovations summarized here describe a shift in the tide of equitable T1D outcomes; however, future work is required to explore additional health equity successes that do not further exacerbate inequities. We also see further opportunities for innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Corresponding author: Ann Mungmode, MPH, T1D Exchange, 11 Avenue de Lafayette, Boston, MA 02111; Email: amungmode@t1dexchange.org

Disclosures: Dr. Ebekozien serve(d) as a director, officer, partner, employee, advisor, consultant, or trustee for the Medtronic Advisory Board and received research grants from Medtronic Diabetes, Eli Lilly, and Dexcom.

Funding: The T1DX-QI is funded by The Leona M. and Harry B. Hemsley Charitable Trust.

From the T1D Exchange, Boston, MA (Ann Mungmode, Nicole Rioles, Jesse Cases, Dr. Ebekozien); The Leona M. and Harry B. Hemsley Charitable Trust, New York, NY (Laurel Koester); and the University of Mississippi School of Population Health, Jackson, MS (Dr. Ebekozien).

Abstract

There have been remarkable innovations in diabetes management since the start of the COVID-19 pandemic, but these groundbreaking innovations are drawing limited focus as the field focuses on the adverse impact of the pandemic on patients with diabetes. This article reviews select population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of the T1D Exchange Quality Improvement Collaborative, a learning health network that focuses on improving care and outcomes for individuals with type 1 diabetes (T1D). Such innovations include expanded telemedicine access, collection of real-world data, machine learning and artificial intelligence, and new diabetes medications and devices. In addition, multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and advocacy efforts for specific populations have been successful. Looking to the future, work is required to explore additional health equity successes that do not further exacerbate inequities and to look for additional innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Keywords: type 1 diabetes, learning health network, continuous glucose monitoring, health equity

One in 10 people in the United States has diabetes.¹ Diabetes is the nation’s second leading cause of death, costing the US health system more than $300 billion annually.² The COVID-19 pandemic presented additional health burdens for people living with diabetes. For example, preexisting diabetes was identified as a risk factor for COVID-19–associated morbidity and mortality.^3,4 Over the past 2 years, there have been remarkable innovations in diabetes management, including stem cell therapy and new medication options. Additionally, improved technology solutions have aided in diabetes management through continuous glucose monitors (CGM), smart insulin pens, advanced hybrid closed-loop systems, and continuous subcutaneous insulin injections.^5,6 Unfortunately, these groundbreaking innovations are drawing limited focus, as the field is rightfully focused on the adverse impact of the pandemic on patients with diabetes.

Learning health networks like the T1D Exchange Quality Improvement Collaborative (T1DX-QI) have implemented some of these innovative solutions to improve care for people with diabetes.⁷ T1DX-QI has more than 50 data-sharing endocrinology centers that care for over 75,000 people with diabetes across the United States (Figure 1). Centers participating in the T1DX-QI use quality improvement (QI) and implementation science methods to quickly translate research into evidence-based clinical practice. T1DX-QI leads diabetes population health and health system research and supports widespread transferability across health care organizations through regular collaborative calls, conferences, and case study documentation.⁸

In this review, we summarize impactful population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of T1DX-QI (see Figure 2 for relevant definitions). This review is limited in scope and is not meant to be an exhaustive list of innovations. The review also reflects significant changes from the perspective of academic diabetes centers, which may not apply to rural or primary care diabetes practices.

The first (A.M.), second (H.H.), and senior (O.E.) authors conducted a scoping review of published literature using terms related to diabetes, population health, and innovation on PubMed Central and Google Scholar for the period March 2020 to June 2022. To complement the review, A.M. and O.E. also reviewed abstracts from presentations at major international diabetes conferences, including the American Diabetes Association (ADA), the International Society for Pediatric and Adolescent Diabetes (ISPAD), the T1DX-QI Learning Session Conference, and the Advanced Technologies & Treatments for Diabetes (ATTD) 2020 to 2022 conferences.^9-14 The authors also searched FDA.gov and ClinicalTrials.gov for relevant insights. A.M. and O.E. sorted the reviewed literature into major themes (Figure 3) from the population health improvement perspective of the T1DX-QI.

Population Health Innovations in Diabetes Management

Expansion of Telemedicine Access

Telemedicine is cost-effective for patients with diabetes,¹⁵ including those with complex cases.¹⁶ Before the COVID-19 pandemic, telemedicine and virtual care were rare in diabetes management. However, the pandemic offered a new opportunity to expand the practice of telemedicine in diabetes management. A study from the T1DX-QI showed that telemedicine visits grew from comprising <1% of visits pre-pandemic (December 2019) to 95.2% during the pandemic (August 2020).¹⁷ Additional studies, like those conducted by Phillip et al,¹⁸ confirmed the noninferiority of telemedicine practice for patients with diabetes.Telemedicine was also found to be an effective strategy to educate patients on the use of diabetes technologies.¹⁹

Real-World Data and Disease Surveillance

As the COVID-19 pandemic exacerbated outcomes for people with type 1 diabetes (T1D), a need arose to understand the immediate effects of the pandemic on people with T1D through real-world data and disease surveillance. In April 2020, the T1DX-QI initiated a multicenter surveillance study to collect data and analyze the impact of COVID-19 on people with T1D. The existing health collaborative served as a springboard for robust surveillance study, documenting numerous works on the effects of COVID-19.^3,4,20-28 Other investigators also embraced the power of real-world surveillance and real-world data.^29,30

Big Data, Machine Learning, and Artificial Intelligence

The past 2 years have seen a shift toward embracing the incredible opportunity to tap the large volume of data generated from routine care for practical insights.³¹ In particular, researchers have demonstrated the widespread application of machine learning and artificial intelligence to improve diabetes management.³² The T1DX-QI also harnessed the growing power of big data by expanding the functionality of innovative benchmarking software. The T1DX QI Portal uses electronic medical record data of diabetes patients for clinic-to-clinic benchmarking and data analysis, using business intelligence solutions.³³

Health Equity

While inequities across various health outcomes have been well documented for years,³⁴ the COVID-19 pandemic further exaggerated racial/ethnic health inequities in T1D.^23,35 In response, several organizations have outlined specific strategies to address these health inequities. Emboldened by the pandemic, the T1DX-QI announced a multipronged approach to address health inequities among patients with T1D through the Health Equity Advancement Lab (HEAL).³⁶ One of HEAL’s main components is using real-world data to champion population-level insights and demonstrate progress in QI efforts.

Multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and these studies are expanding our understanding of the chasm.³⁷ There have also been innovative solutions to addressing these inequities, with multiple studies published over the past 2 years.³⁸ A source of inequity among patients with T1D is the lack of representation of racial/ethnic minorities with T1D in clinical trials.³⁹ The T1DX-QI suggests that the equity-adapted framework for QI can be applied by research leaders to support trial diversity and representation, ensuring future device innovations are meaningful for all people with T1D.⁴⁰

Diabetes Devices

Glucose monitoring and insulin therapy are vital tools to support individuals living with T1D, and devices such as CGM and insulin pumps have become the standard of care for diabetes management (Table).⁴¹ Innovations in diabetes technology and device access are imperative for a chronic disease with no cure.

The COVID-19 pandemic created an opportunity to increase access to diabetes devices in inpatient settings. In 2020, the US Food and Drug Administration expanded the use of CGM to support remote monitoring of patients in inpatient hospital settings, simultaneously supporting the glucose monitoring needs of patients with T1D and reducing COVID-19 transmission through reduced patient-clinician contact.⁴² This effort has been expanded and will continue in 2022 and beyond,⁴³ and aligns with the growing consensus that supports patients wearing both CGMs and insulin pumps in ambulatory settings to improve patient health outcomes.⁴⁴

Since 2020, innovations in diabetes technology have improved and increased the variety of options available to people with T1D and made them easier to use (Table). New, advanced hybrid closed-loop systems have progressed to offer Bluetooth features, including automatic software upgrades, tubeless systems, and the ability to allow parents to use their smartphones to bolus for children.^45-47 The next big step in insulin delivery innovation is the release of functioning, fully closed loop systems, of which several are currently in clinical trials.⁴⁸ These systems support reduced hypoglycemia and improved time in range.⁴⁹

Additional innovations in insulin delivery have improved the user experience and expanded therapeutic options, including a variety of smart insulin pens complete with dosing logs^50,51 and even a patch to deliver insulin without the burden of injections.⁵² As barriers to diabetes technology persist,⁵³ innovations in alternate insulin delivery provide people with T1D more options to align with their personal access and technology preferences.

Innovations in CGM address cited barriers to their use, including size or overall wear.^53-55 CGMs released in the past few years are smaller in physical size, have longer durations of time between changings, are more accurate, and do not require calibrations for accuracy.

New Diabetes Medications

Many new medications and therapeutic advances have become available in the past 2 years.⁵⁶ Additionally, more medications are being tested as adjunct therapies to support glycemic management in patients with T1D, including metformin, sodium-glucose cotransporter 1 and 2 inhibitors, pramlintide, glucagon-like polypeptide-1 analogs, and glucagon receptor agonists.⁵⁷ Other recent advances include stem cell replacement therapy for patients with T1D.⁵⁸ The ultra-long-acting biosimilar insulins are one medical innovation that has been stalled, rather than propelled, during the COVID-19 pandemic.⁵⁹

Diabetes Policy Advocacy

People with T1D require insulin to survive. The cost of insulin has increased in recent years, with some studies citing a 64% to 100% increase in the past decade.^60,61 In fact, 1 in 4 insulin users report that cost has impacted their insulin use, including rationing their insulin.⁶² Lockdowns during the COVID-19 pandemic stressed US families financially, increasing the urgency for insulin cost caps.

Although the COVID-19 pandemic halted national conversations on drug financing,⁶³ advocacy efforts have succeeded for specific populations. The new Medicare Part D Senior Savings Model will cap the cost of insulin at $35 for a 30-day supply,⁶⁴ and 20 states passed legislation capping insulin pricing.⁶² Efforts to codify national cost caps are under debate, including the passage of the Affordable Insulin Now Act, which passed the House in March 2022 and is currently under review in the Senate.⁶⁵

Funders and industry partners play a crucial role in leading and supporting innovations that improve the lives of people with T1D and reduce society’s costs of living with the disease. Data infrastructure is critical to supporting population health. While building the data infrastructure to support population health is both time- and resource-intensive, private foundations such as Helmsley are uniquely positioned—and have a responsibility—to take large, informed risks to help reach all communities with T1D.

The T1DX-QI is the largest source of population health data on T1D in the United States and is becoming the premiere data authority on its incidence, prevalence, and outcomes. The T1DX-QI enables a robust understanding of T1D-related health trends at the population level, as well as trends among clinics and providers. Pilot centers in the T1DX-QI have reported reductions in patients’ A1c and acute diabetes-related events, as well as improvements in device usage and depression screening. The ability to capture changes speaks to the promise and power of these data to demonstrate the clinical impact of QI interventions and to support the spread of best practices and learnings across health systems.

Additional philanthropic efforts have supported innovation in the last 2 years. For example, the JDRF, a nonprofit philanthropic equity firm, has supported efforts in developing artificial pancreas systems and cell therapies currently in clinical trials like teplizumab, a drug that has demonstrated delayed onset of T1D through JDRF’s T1D Fund.⁶⁶ Industry partners also have an opportunity for significant influence in this area, as they continue to fund meaningful projects to advance care for people with T1D.⁶⁷

Conclusion

We are optimistic that the innovations summarized here describe a shift in the tide of equitable T1D outcomes; however, future work is required to explore additional health equity successes that do not further exacerbate inequities. We also see further opportunities for innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Corresponding author: Ann Mungmode, MPH, T1D Exchange, 11 Avenue de Lafayette, Boston, MA 02111; Email: amungmode@t1dexchange.org

Disclosures: Dr. Ebekozien serve(d) as a director, officer, partner, employee, advisor, consultant, or trustee for the Medtronic Advisory Board and received research grants from Medtronic Diabetes, Eli Lilly, and Dexcom.

Funding: The T1DX-QI is funded by The Leona M. and Harry B. Hemsley Charitable Trust.

From the T1D Exchange, Boston, MA (Ann Mungmode, Nicole Rioles, Jesse Cases, Dr. Ebekozien); The Leona M. and Harry B. Hemsley Charitable Trust, New York, NY (Laurel Koester); and the University of Mississippi School of Population Health, Jackson, MS (Dr. Ebekozien).

Abstract

There have been remarkable innovations in diabetes management since the start of the COVID-19 pandemic, but these groundbreaking innovations are drawing limited focus as the field focuses on the adverse impact of the pandemic on patients with diabetes. This article reviews select population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of the T1D Exchange Quality Improvement Collaborative, a learning health network that focuses on improving care and outcomes for individuals with type 1 diabetes (T1D). Such innovations include expanded telemedicine access, collection of real-world data, machine learning and artificial intelligence, and new diabetes medications and devices. In addition, multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and advocacy efforts for specific populations have been successful. Looking to the future, work is required to explore additional health equity successes that do not further exacerbate inequities and to look for additional innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Keywords: type 1 diabetes, learning health network, continuous glucose monitoring, health equity

One in 10 people in the United States has diabetes.¹ Diabetes is the nation’s second leading cause of death, costing the US health system more than $300 billion annually.² The COVID-19 pandemic presented additional health burdens for people living with diabetes. For example, preexisting diabetes was identified as a risk factor for COVID-19–associated morbidity and mortality.^3,4 Over the past 2 years, there have been remarkable innovations in diabetes management, including stem cell therapy and new medication options. Additionally, improved technology solutions have aided in diabetes management through continuous glucose monitors (CGM), smart insulin pens, advanced hybrid closed-loop systems, and continuous subcutaneous insulin injections.^5,6 Unfortunately, these groundbreaking innovations are drawing limited focus, as the field is rightfully focused on the adverse impact of the pandemic on patients with diabetes.

Learning health networks like the T1D Exchange Quality Improvement Collaborative (T1DX-QI) have implemented some of these innovative solutions to improve care for people with diabetes.⁷ T1DX-QI has more than 50 data-sharing endocrinology centers that care for over 75,000 people with diabetes across the United States (Figure 1). Centers participating in the T1DX-QI use quality improvement (QI) and implementation science methods to quickly translate research into evidence-based clinical practice. T1DX-QI leads diabetes population health and health system research and supports widespread transferability across health care organizations through regular collaborative calls, conferences, and case study documentation.⁸

In this review, we summarize impactful population health innovations in diabetes management that have become available over the past 2 years of the COVID-19 pandemic from the perspective of T1DX-QI (see Figure 2 for relevant definitions). This review is limited in scope and is not meant to be an exhaustive list of innovations. The review also reflects significant changes from the perspective of academic diabetes centers, which may not apply to rural or primary care diabetes practices.

The first (A.M.), second (H.H.), and senior (O.E.) authors conducted a scoping review of published literature using terms related to diabetes, population health, and innovation on PubMed Central and Google Scholar for the period March 2020 to June 2022. To complement the review, A.M. and O.E. also reviewed abstracts from presentations at major international diabetes conferences, including the American Diabetes Association (ADA), the International Society for Pediatric and Adolescent Diabetes (ISPAD), the T1DX-QI Learning Session Conference, and the Advanced Technologies & Treatments for Diabetes (ATTD) 2020 to 2022 conferences.^9-14 The authors also searched FDA.gov and ClinicalTrials.gov for relevant insights. A.M. and O.E. sorted the reviewed literature into major themes (Figure 3) from the population health improvement perspective of the T1DX-QI.

Population Health Innovations in Diabetes Management

Expansion of Telemedicine Access

Telemedicine is cost-effective for patients with diabetes,¹⁵ including those with complex cases.¹⁶ Before the COVID-19 pandemic, telemedicine and virtual care were rare in diabetes management. However, the pandemic offered a new opportunity to expand the practice of telemedicine in diabetes management. A study from the T1DX-QI showed that telemedicine visits grew from comprising <1% of visits pre-pandemic (December 2019) to 95.2% during the pandemic (August 2020).¹⁷ Additional studies, like those conducted by Phillip et al,¹⁸ confirmed the noninferiority of telemedicine practice for patients with diabetes.Telemedicine was also found to be an effective strategy to educate patients on the use of diabetes technologies.¹⁹

Real-World Data and Disease Surveillance

As the COVID-19 pandemic exacerbated outcomes for people with type 1 diabetes (T1D), a need arose to understand the immediate effects of the pandemic on people with T1D through real-world data and disease surveillance. In April 2020, the T1DX-QI initiated a multicenter surveillance study to collect data and analyze the impact of COVID-19 on people with T1D. The existing health collaborative served as a springboard for robust surveillance study, documenting numerous works on the effects of COVID-19.^3,4,20-28 Other investigators also embraced the power of real-world surveillance and real-world data.^29,30

Big Data, Machine Learning, and Artificial Intelligence

The past 2 years have seen a shift toward embracing the incredible opportunity to tap the large volume of data generated from routine care for practical insights.³¹ In particular, researchers have demonstrated the widespread application of machine learning and artificial intelligence to improve diabetes management.³² The T1DX-QI also harnessed the growing power of big data by expanding the functionality of innovative benchmarking software. The T1DX QI Portal uses electronic medical record data of diabetes patients for clinic-to-clinic benchmarking and data analysis, using business intelligence solutions.³³

Health Equity

While inequities across various health outcomes have been well documented for years,³⁴ the COVID-19 pandemic further exaggerated racial/ethnic health inequities in T1D.^23,35 In response, several organizations have outlined specific strategies to address these health inequities. Emboldened by the pandemic, the T1DX-QI announced a multipronged approach to address health inequities among patients with T1D through the Health Equity Advancement Lab (HEAL).³⁶ One of HEAL’s main components is using real-world data to champion population-level insights and demonstrate progress in QI efforts.

Multiple innovative studies have been undertaken to explore contributors to health inequities in diabetes, and these studies are expanding our understanding of the chasm.³⁷ There have also been innovative solutions to addressing these inequities, with multiple studies published over the past 2 years.³⁸ A source of inequity among patients with T1D is the lack of representation of racial/ethnic minorities with T1D in clinical trials.³⁹ The T1DX-QI suggests that the equity-adapted framework for QI can be applied by research leaders to support trial diversity and representation, ensuring future device innovations are meaningful for all people with T1D.⁴⁰

Diabetes Devices

Glucose monitoring and insulin therapy are vital tools to support individuals living with T1D, and devices such as CGM and insulin pumps have become the standard of care for diabetes management (Table).⁴¹ Innovations in diabetes technology and device access are imperative for a chronic disease with no cure.

The COVID-19 pandemic created an opportunity to increase access to diabetes devices in inpatient settings. In 2020, the US Food and Drug Administration expanded the use of CGM to support remote monitoring of patients in inpatient hospital settings, simultaneously supporting the glucose monitoring needs of patients with T1D and reducing COVID-19 transmission through reduced patient-clinician contact.⁴² This effort has been expanded and will continue in 2022 and beyond,⁴³ and aligns with the growing consensus that supports patients wearing both CGMs and insulin pumps in ambulatory settings to improve patient health outcomes.⁴⁴

Since 2020, innovations in diabetes technology have improved and increased the variety of options available to people with T1D and made them easier to use (Table). New, advanced hybrid closed-loop systems have progressed to offer Bluetooth features, including automatic software upgrades, tubeless systems, and the ability to allow parents to use their smartphones to bolus for children.^45-47 The next big step in insulin delivery innovation is the release of functioning, fully closed loop systems, of which several are currently in clinical trials.⁴⁸ These systems support reduced hypoglycemia and improved time in range.⁴⁹

Additional innovations in insulin delivery have improved the user experience and expanded therapeutic options, including a variety of smart insulin pens complete with dosing logs^50,51 and even a patch to deliver insulin without the burden of injections.⁵² As barriers to diabetes technology persist,⁵³ innovations in alternate insulin delivery provide people with T1D more options to align with their personal access and technology preferences.

Innovations in CGM address cited barriers to their use, including size or overall wear.^53-55 CGMs released in the past few years are smaller in physical size, have longer durations of time between changings, are more accurate, and do not require calibrations for accuracy.

New Diabetes Medications

Many new medications and therapeutic advances have become available in the past 2 years.⁵⁶ Additionally, more medications are being tested as adjunct therapies to support glycemic management in patients with T1D, including metformin, sodium-glucose cotransporter 1 and 2 inhibitors, pramlintide, glucagon-like polypeptide-1 analogs, and glucagon receptor agonists.⁵⁷ Other recent advances include stem cell replacement therapy for patients with T1D.⁵⁸ The ultra-long-acting biosimilar insulins are one medical innovation that has been stalled, rather than propelled, during the COVID-19 pandemic.⁵⁹

Diabetes Policy Advocacy

People with T1D require insulin to survive. The cost of insulin has increased in recent years, with some studies citing a 64% to 100% increase in the past decade.^60,61 In fact, 1 in 4 insulin users report that cost has impacted their insulin use, including rationing their insulin.⁶² Lockdowns during the COVID-19 pandemic stressed US families financially, increasing the urgency for insulin cost caps.

Although the COVID-19 pandemic halted national conversations on drug financing,⁶³ advocacy efforts have succeeded for specific populations. The new Medicare Part D Senior Savings Model will cap the cost of insulin at $35 for a 30-day supply,⁶⁴ and 20 states passed legislation capping insulin pricing.⁶² Efforts to codify national cost caps are under debate, including the passage of the Affordable Insulin Now Act, which passed the House in March 2022 and is currently under review in the Senate.⁶⁵

Funders and industry partners play a crucial role in leading and supporting innovations that improve the lives of people with T1D and reduce society’s costs of living with the disease. Data infrastructure is critical to supporting population health. While building the data infrastructure to support population health is both time- and resource-intensive, private foundations such as Helmsley are uniquely positioned—and have a responsibility—to take large, informed risks to help reach all communities with T1D.

The T1DX-QI is the largest source of population health data on T1D in the United States and is becoming the premiere data authority on its incidence, prevalence, and outcomes. The T1DX-QI enables a robust understanding of T1D-related health trends at the population level, as well as trends among clinics and providers. Pilot centers in the T1DX-QI have reported reductions in patients’ A1c and acute diabetes-related events, as well as improvements in device usage and depression screening. The ability to capture changes speaks to the promise and power of these data to demonstrate the clinical impact of QI interventions and to support the spread of best practices and learnings across health systems.

Additional philanthropic efforts have supported innovation in the last 2 years. For example, the JDRF, a nonprofit philanthropic equity firm, has supported efforts in developing artificial pancreas systems and cell therapies currently in clinical trials like teplizumab, a drug that has demonstrated delayed onset of T1D through JDRF’s T1D Fund.⁶⁶ Industry partners also have an opportunity for significant influence in this area, as they continue to fund meaningful projects to advance care for people with T1D.⁶⁷

Conclusion

We are optimistic that the innovations summarized here describe a shift in the tide of equitable T1D outcomes; however, future work is required to explore additional health equity successes that do not further exacerbate inequities. We also see further opportunities for innovative ways to engage people with T1D in their health care through conversations on social determinants of health and societal structures.

Corresponding author: Ann Mungmode, MPH, T1D Exchange, 11 Avenue de Lafayette, Boston, MA 02111; Email: amungmode@t1dexchange.org

Disclosures: Dr. Ebekozien serve(d) as a director, officer, partner, employee, advisor, consultant, or trustee for the Medtronic Advisory Board and received research grants from Medtronic Diabetes, Eli Lilly, and Dexcom.

Funding: The T1DX-QI is funded by The Leona M. and Harry B. Hemsley Charitable Trust.

Study 1 Overview (Bayliss et al)

Objective: To examine the effect of a deprescribing educational intervention on medication use in older adults with cognitive impairment.

Design: This was a pragmatic, cluster randomized trial conducted in 8 primary care clinics that are part of a nonprofit health care system.

Setting and participants: The primary care clinic populations ranged from 170 to 1125 patients per clinic. The primary care clinics were randomly assigned to intervention or control using a uniform distribution in blocks by clinic size. Eligibility criteria for participants at those practices included age 65 years or older; health plan enrollment at least 1 year prior to intervention; diagnosis of Alzheimer disease and related dementia (ADRD) or mild cognitive impairment (MCI) by International Statistical Classification of Diseases and Related Health Problems, Tenth Revision code or from problem list; 1 or more chronic conditions from those in the Chronic Conditions Warehouse; and 5 or more long-term medications. Those who scheduled a visit at their primary care clinic in advance were eligible for the intervention. Primary care clinicians in intervention clinics were eligible to receive the clinician portion of the intervention. A total of 1433 participants were enrolled in the intervention group, and 1579 participants were enrolled in the control group.

Intervention: The intervention included 2 components: a patient and family component with materials mailed in advance of their primary care visits and a clinician component comprising monthly educational materials on deprescribing and notification in the electronic health record about visits with patient participants. The patient and family component consisted of a brochure titled “Managing Medication” and a questionnaire on attitudes toward deprescribing intended to educate patients and family about deprescribing. Clinicians at intervention clinics received an educational presentation at a monthly clinician meeting as well as tip sheets and a poster on deprescribing topics, and they also were notified of upcoming appointments with patients who received the patient component of the intervention. For the control group, patients and family did not receive any materials, and clinicians did not receive intervention materials or notification of participants enrolled in the trial. Usual care in both intervention and control groups included medication reconciliation and electronic health record alerts for potentially high-risk medications.

Main outcome measures: The primary outcomes of the study were the number of long-term medications per individual and the proportion of patients prescribed 1 or more potentially inappropriate medications. Outcome measurements were extracted from the electronic clinical data, and outcomes were assessed at 6 months, which involved comparing counts of medications at baseline with medications at 6 months. Long-term medications were defined as medications that are prescribed for 28 days or more based on pharmacy dispensing data. Potentially inappropriate medications (PIMs) were defined using the Beers list of medications to avoid in those with cognitive impairment and opioid medications. Analyses were conducted as intention to treat.

Main results: In the intervention group and control group, 56.2% and 54.4% of participants were women, and the mean age was 80.1 years (SD, 7.2) and 79.9 years (SD, 7.5), respectively. At baseline, the mean number of long-term medications was 7.0 (SD, 2.1) in the intervention group and 7.0 (SD, 2.2) in the control group. The proportion of patients taking any PIMs was 30.5% in the intervention group and 29.6% in the control group. At 6 months, the mean number of long-term medications was 6.4 in the intervention group and 6.5 in the control group, with an adjusted difference of –0.1 (95% CI, –0.2 to 0.04; P = .14); the proportion of patients with any PIMs was 17.8% in the intervention group and 20.9% in the control group, with an adjusted difference of –3.2% (95% CI, –6.2 to 0.4; P = .08). Preplanned analyses to examine subgroup differences for those with a higher number of medications (7+ vs 5 or 6 medications) did not find different effects of the intervention.

Conclusion: This educational intervention on deprescribing did not result in reductions in the number of medications or the use of PIMs in patients with cognitive impairment.

Study 2 Overview (Gedde et al)

Objective: To examine the effect of a deprescribing intervention (COSMOS) on medication use for nursing home residents.

Design: This was a randomized clinical trial.

Setting and participants: This trial was conducted in 67 units in 33 nursing homes in Norway. Participants were nursing home residents recruited from August 2014 to March 2015. Inclusion criteria included adults aged 65 years and older with at least 2 years of residency in nursing homes. Exclusion criteria included diagnosis of schizophrenia and a life expectancy of 6 months or less. Participants were followed for 4 months; participants were considered lost to follow-up if they died or moved from the nursing home unit. The analyses were per protocol and did not include those lost to follow-up or those who did not undergo a medication review in the intervention group. A total of 217 and 211 residents were included in the intervention and control groups, respectively.

Intervention: The intervention contained 5 components: communication and advance care planning, systematic pain management, medication reviews with collegial mentoring, organization of activities adjusted to needs and preferences, and safety. For medication review, the nursing home physician reviewed medications together with a nurse and study physicians who provided mentoring. The medication review involved a structured process that used assessment tools for behavioral and psychological symptoms of dementia (BPSD), activities of daily living (ADL), pain, cognitive status, well-being and quality of life, and clinical metrics of blood pressure, pulse, and body mass index. The study utilized the START/STOPP criteria¹ for medication use in addition to a list of medications with anticholinergic properties for the medication review. In addition, drug interactions were documented through a drug interaction database; the team also incorporated patient wishes and concerns in the medication reviews. The nursing home physician made final decisions on medications. For the control group, nursing home residents received usual care without this intervention.

Main outcome measures: The primary outcome of the study was the mean change in the number of prescribed psychotropic medications, both regularly scheduled and total medications (which also included on-demand drugs) received at 4 months when compared to baseline. Psychotropic medications included antipsychotics, anxiolytics, hypnotics or sedatives, antidepressants, and antidementia drugs. Secondary outcomes included mean changes in BPSD using the Neuropsychiatric Inventory-Nursing home version (NPI-NH) and the Cornell Scale for Depression for Dementia (CSDD) and ADL using the Physical Self Maintenance Scale (PSMS).

Main results: In both the intervention and control groups, 76% of participants were women, and mean age was 86.3 years (SD, 7.95) in the intervention group and 86.6 years (SD, 7.21) in the control group. At baseline, the mean number of total medications was 10.9 (SD, 4.6) in the intervention group and 10.9 (SD, 4.7) in the control group, and the mean number of psychotropic medications was 2.2 (SD, 1.6) and 2.2 (SD, 1.7) in the intervention and control groups, respectively. At 4 months, the mean change from baseline of total psychotropic medications was –0.34 in the intervention group and 0.01 in the control group (P < .001), and the mean change of regularly scheduled psychotropic medications was –0.21 in the intervention group and 0.02 in the control group (P < .001). Measures of BPSD and depression did not differ between intervention and control groups, and ADL showed a small improvement in the intervention group.

Conclusion: This intervention reduced the use of psychotropic medications in nursing home residents without worsening BPSD or depression and may have yielded improvements in ADL.

Commentary

Polypharmacy is common among older adults, as many of them have multiple chronic conditions and often take multiple medications for managing them. Polypharmacy increases the risk of drug interactions and adverse effects from medications; older adults who are frail and/or who have cognitive impairment are especially at risk. Reducing medication use, especially medications likely to cause adverse effects such as those with anticholinergic properties, has the potential to yield beneficial effects while reducing the burden of taking medications. A large randomized trial found that a pharmacist-led education intervention can be effective in reducing PIM use in community-dwelling older adults,² and that targeting patient motivation and capacity to deprescribe could be effective.³ This study by Bayliss and colleagues (Study 1), however, fell short of the effects seen in the earlier D-PRESCRIBE trial. One of the reasons for these findings may be that the clinician portion of the intervention was less intensive than that used in the earlier trial; specifically, in the present study, clinicians were not provided with or expected to utilize tools for structured medication review or deprescribing. Although the intervention primes the patient and family for discussions around deprescribing through the use of a brochure and questionnaire, the clinician portion of the intervention was less structured. Another example of an effective intervention that provided a more structured deprescribing intervention beyond education of clinicians utilized electronic decision-support to assist with deprescribing.⁴

The findings from the Gedde et al study (Study 2) are comparable to those of prior studies in the nursing home population,⁵ where participants are likely to take a large number of medications, including psychotropic medications, and are more likely to be frail. However, Gedde and colleagues employed a bundled intervention⁶ that included other components besides medication review, and thus it is unclear whether the effect on ADL can be attributed to the deprescribing of medications alone. Gedde et al’s finding that deprescribing can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression is an important contribution to our knowledge about polypharmacy and deprescribing in older patients. Thus, nursing home residents, their families, and clinicians could expect that the deprescribing of psychotropic medications does not lead to worsening symptoms. Of note, the clinician portion of the intervention in the Gedde et al study was quite structured, and this structure may have contributed to the observed effects.

Applications for Clinical Practice and System Implementation

Both studies add to the literature on deprescribing and may offer options for researchers and clinicians who are considering potential components of an effective deprescribing intervention. Patient activation for deprescribing via the methods used in these 2 studies may help to prime patients for conversations about deprescribing; however, as shown by the Bayliss et al study, a more structured approach to clinical encounters may be needed when deprescribing, such as the use of tools in the electronic health record, in order to reduce the use of medication deemed unnecessary or potentially harmful. Further studies should examine the effect of deprescribing on medication use, but perhaps even more importantly, how deprescribing impacts patient outcomes both in terms of risks and benefits.

Practice Points

• A more structured approach to clinical encounters (eg, the use of tools in the electronic health record) may be needed when deprescribing unnecessary or potentially harmful medications in older patients in community settings.
• In the nursing home setting, structured deprescribing intervention can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression.

–William W. Hung, MD, MPH

Study 1 Overview (Bayliss et al)

Objective: To examine the effect of a deprescribing educational intervention on medication use in older adults with cognitive impairment.

Design: This was a pragmatic, cluster randomized trial conducted in 8 primary care clinics that are part of a nonprofit health care system.

Setting and participants: The primary care clinic populations ranged from 170 to 1125 patients per clinic. The primary care clinics were randomly assigned to intervention or control using a uniform distribution in blocks by clinic size. Eligibility criteria for participants at those practices included age 65 years or older; health plan enrollment at least 1 year prior to intervention; diagnosis of Alzheimer disease and related dementia (ADRD) or mild cognitive impairment (MCI) by International Statistical Classification of Diseases and Related Health Problems, Tenth Revision code or from problem list; 1 or more chronic conditions from those in the Chronic Conditions Warehouse; and 5 or more long-term medications. Those who scheduled a visit at their primary care clinic in advance were eligible for the intervention. Primary care clinicians in intervention clinics were eligible to receive the clinician portion of the intervention. A total of 1433 participants were enrolled in the intervention group, and 1579 participants were enrolled in the control group.

Intervention: The intervention included 2 components: a patient and family component with materials mailed in advance of their primary care visits and a clinician component comprising monthly educational materials on deprescribing and notification in the electronic health record about visits with patient participants. The patient and family component consisted of a brochure titled “Managing Medication” and a questionnaire on attitudes toward deprescribing intended to educate patients and family about deprescribing. Clinicians at intervention clinics received an educational presentation at a monthly clinician meeting as well as tip sheets and a poster on deprescribing topics, and they also were notified of upcoming appointments with patients who received the patient component of the intervention. For the control group, patients and family did not receive any materials, and clinicians did not receive intervention materials or notification of participants enrolled in the trial. Usual care in both intervention and control groups included medication reconciliation and electronic health record alerts for potentially high-risk medications.

Main outcome measures: The primary outcomes of the study were the number of long-term medications per individual and the proportion of patients prescribed 1 or more potentially inappropriate medications. Outcome measurements were extracted from the electronic clinical data, and outcomes were assessed at 6 months, which involved comparing counts of medications at baseline with medications at 6 months. Long-term medications were defined as medications that are prescribed for 28 days or more based on pharmacy dispensing data. Potentially inappropriate medications (PIMs) were defined using the Beers list of medications to avoid in those with cognitive impairment and opioid medications. Analyses were conducted as intention to treat.

Main results: In the intervention group and control group, 56.2% and 54.4% of participants were women, and the mean age was 80.1 years (SD, 7.2) and 79.9 years (SD, 7.5), respectively. At baseline, the mean number of long-term medications was 7.0 (SD, 2.1) in the intervention group and 7.0 (SD, 2.2) in the control group. The proportion of patients taking any PIMs was 30.5% in the intervention group and 29.6% in the control group. At 6 months, the mean number of long-term medications was 6.4 in the intervention group and 6.5 in the control group, with an adjusted difference of –0.1 (95% CI, –0.2 to 0.04; P = .14); the proportion of patients with any PIMs was 17.8% in the intervention group and 20.9% in the control group, with an adjusted difference of –3.2% (95% CI, –6.2 to 0.4; P = .08). Preplanned analyses to examine subgroup differences for those with a higher number of medications (7+ vs 5 or 6 medications) did not find different effects of the intervention.

Conclusion: This educational intervention on deprescribing did not result in reductions in the number of medications or the use of PIMs in patients with cognitive impairment.

Study 2 Overview (Gedde et al)

Objective: To examine the effect of a deprescribing intervention (COSMOS) on medication use for nursing home residents.

Design: This was a randomized clinical trial.

Setting and participants: This trial was conducted in 67 units in 33 nursing homes in Norway. Participants were nursing home residents recruited from August 2014 to March 2015. Inclusion criteria included adults aged 65 years and older with at least 2 years of residency in nursing homes. Exclusion criteria included diagnosis of schizophrenia and a life expectancy of 6 months or less. Participants were followed for 4 months; participants were considered lost to follow-up if they died or moved from the nursing home unit. The analyses were per protocol and did not include those lost to follow-up or those who did not undergo a medication review in the intervention group. A total of 217 and 211 residents were included in the intervention and control groups, respectively.

Intervention: The intervention contained 5 components: communication and advance care planning, systematic pain management, medication reviews with collegial mentoring, organization of activities adjusted to needs and preferences, and safety. For medication review, the nursing home physician reviewed medications together with a nurse and study physicians who provided mentoring. The medication review involved a structured process that used assessment tools for behavioral and psychological symptoms of dementia (BPSD), activities of daily living (ADL), pain, cognitive status, well-being and quality of life, and clinical metrics of blood pressure, pulse, and body mass index. The study utilized the START/STOPP criteria¹ for medication use in addition to a list of medications with anticholinergic properties for the medication review. In addition, drug interactions were documented through a drug interaction database; the team also incorporated patient wishes and concerns in the medication reviews. The nursing home physician made final decisions on medications. For the control group, nursing home residents received usual care without this intervention.

Main outcome measures: The primary outcome of the study was the mean change in the number of prescribed psychotropic medications, both regularly scheduled and total medications (which also included on-demand drugs) received at 4 months when compared to baseline. Psychotropic medications included antipsychotics, anxiolytics, hypnotics or sedatives, antidepressants, and antidementia drugs. Secondary outcomes included mean changes in BPSD using the Neuropsychiatric Inventory-Nursing home version (NPI-NH) and the Cornell Scale for Depression for Dementia (CSDD) and ADL using the Physical Self Maintenance Scale (PSMS).

Main results: In both the intervention and control groups, 76% of participants were women, and mean age was 86.3 years (SD, 7.95) in the intervention group and 86.6 years (SD, 7.21) in the control group. At baseline, the mean number of total medications was 10.9 (SD, 4.6) in the intervention group and 10.9 (SD, 4.7) in the control group, and the mean number of psychotropic medications was 2.2 (SD, 1.6) and 2.2 (SD, 1.7) in the intervention and control groups, respectively. At 4 months, the mean change from baseline of total psychotropic medications was –0.34 in the intervention group and 0.01 in the control group (P < .001), and the mean change of regularly scheduled psychotropic medications was –0.21 in the intervention group and 0.02 in the control group (P < .001). Measures of BPSD and depression did not differ between intervention and control groups, and ADL showed a small improvement in the intervention group.

Conclusion: This intervention reduced the use of psychotropic medications in nursing home residents without worsening BPSD or depression and may have yielded improvements in ADL.

Commentary

Polypharmacy is common among older adults, as many of them have multiple chronic conditions and often take multiple medications for managing them. Polypharmacy increases the risk of drug interactions and adverse effects from medications; older adults who are frail and/or who have cognitive impairment are especially at risk. Reducing medication use, especially medications likely to cause adverse effects such as those with anticholinergic properties, has the potential to yield beneficial effects while reducing the burden of taking medications. A large randomized trial found that a pharmacist-led education intervention can be effective in reducing PIM use in community-dwelling older adults,² and that targeting patient motivation and capacity to deprescribe could be effective.³ This study by Bayliss and colleagues (Study 1), however, fell short of the effects seen in the earlier D-PRESCRIBE trial. One of the reasons for these findings may be that the clinician portion of the intervention was less intensive than that used in the earlier trial; specifically, in the present study, clinicians were not provided with or expected to utilize tools for structured medication review or deprescribing. Although the intervention primes the patient and family for discussions around deprescribing through the use of a brochure and questionnaire, the clinician portion of the intervention was less structured. Another example of an effective intervention that provided a more structured deprescribing intervention beyond education of clinicians utilized electronic decision-support to assist with deprescribing.⁴

The findings from the Gedde et al study (Study 2) are comparable to those of prior studies in the nursing home population,⁵ where participants are likely to take a large number of medications, including psychotropic medications, and are more likely to be frail. However, Gedde and colleagues employed a bundled intervention⁶ that included other components besides medication review, and thus it is unclear whether the effect on ADL can be attributed to the deprescribing of medications alone. Gedde et al’s finding that deprescribing can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression is an important contribution to our knowledge about polypharmacy and deprescribing in older patients. Thus, nursing home residents, their families, and clinicians could expect that the deprescribing of psychotropic medications does not lead to worsening symptoms. Of note, the clinician portion of the intervention in the Gedde et al study was quite structured, and this structure may have contributed to the observed effects.

Applications for Clinical Practice and System Implementation

Both studies add to the literature on deprescribing and may offer options for researchers and clinicians who are considering potential components of an effective deprescribing intervention. Patient activation for deprescribing via the methods used in these 2 studies may help to prime patients for conversations about deprescribing; however, as shown by the Bayliss et al study, a more structured approach to clinical encounters may be needed when deprescribing, such as the use of tools in the electronic health record, in order to reduce the use of medication deemed unnecessary or potentially harmful. Further studies should examine the effect of deprescribing on medication use, but perhaps even more importantly, how deprescribing impacts patient outcomes both in terms of risks and benefits.

Practice Points

• A more structured approach to clinical encounters (eg, the use of tools in the electronic health record) may be needed when deprescribing unnecessary or potentially harmful medications in older patients in community settings.
• In the nursing home setting, structured deprescribing intervention can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression.

–William W. Hung, MD, MPH

Study 1 Overview (Bayliss et al)

Objective: To examine the effect of a deprescribing educational intervention on medication use in older adults with cognitive impairment.

Design: This was a pragmatic, cluster randomized trial conducted in 8 primary care clinics that are part of a nonprofit health care system.

Setting and participants: The primary care clinic populations ranged from 170 to 1125 patients per clinic. The primary care clinics were randomly assigned to intervention or control using a uniform distribution in blocks by clinic size. Eligibility criteria for participants at those practices included age 65 years or older; health plan enrollment at least 1 year prior to intervention; diagnosis of Alzheimer disease and related dementia (ADRD) or mild cognitive impairment (MCI) by International Statistical Classification of Diseases and Related Health Problems, Tenth Revision code or from problem list; 1 or more chronic conditions from those in the Chronic Conditions Warehouse; and 5 or more long-term medications. Those who scheduled a visit at their primary care clinic in advance were eligible for the intervention. Primary care clinicians in intervention clinics were eligible to receive the clinician portion of the intervention. A total of 1433 participants were enrolled in the intervention group, and 1579 participants were enrolled in the control group.

Intervention: The intervention included 2 components: a patient and family component with materials mailed in advance of their primary care visits and a clinician component comprising monthly educational materials on deprescribing and notification in the electronic health record about visits with patient participants. The patient and family component consisted of a brochure titled “Managing Medication” and a questionnaire on attitudes toward deprescribing intended to educate patients and family about deprescribing. Clinicians at intervention clinics received an educational presentation at a monthly clinician meeting as well as tip sheets and a poster on deprescribing topics, and they also were notified of upcoming appointments with patients who received the patient component of the intervention. For the control group, patients and family did not receive any materials, and clinicians did not receive intervention materials or notification of participants enrolled in the trial. Usual care in both intervention and control groups included medication reconciliation and electronic health record alerts for potentially high-risk medications.

Main outcome measures: The primary outcomes of the study were the number of long-term medications per individual and the proportion of patients prescribed 1 or more potentially inappropriate medications. Outcome measurements were extracted from the electronic clinical data, and outcomes were assessed at 6 months, which involved comparing counts of medications at baseline with medications at 6 months. Long-term medications were defined as medications that are prescribed for 28 days or more based on pharmacy dispensing data. Potentially inappropriate medications (PIMs) were defined using the Beers list of medications to avoid in those with cognitive impairment and opioid medications. Analyses were conducted as intention to treat.

Main results: In the intervention group and control group, 56.2% and 54.4% of participants were women, and the mean age was 80.1 years (SD, 7.2) and 79.9 years (SD, 7.5), respectively. At baseline, the mean number of long-term medications was 7.0 (SD, 2.1) in the intervention group and 7.0 (SD, 2.2) in the control group. The proportion of patients taking any PIMs was 30.5% in the intervention group and 29.6% in the control group. At 6 months, the mean number of long-term medications was 6.4 in the intervention group and 6.5 in the control group, with an adjusted difference of –0.1 (95% CI, –0.2 to 0.04; P = .14); the proportion of patients with any PIMs was 17.8% in the intervention group and 20.9% in the control group, with an adjusted difference of –3.2% (95% CI, –6.2 to 0.4; P = .08). Preplanned analyses to examine subgroup differences for those with a higher number of medications (7+ vs 5 or 6 medications) did not find different effects of the intervention.

Conclusion: This educational intervention on deprescribing did not result in reductions in the number of medications or the use of PIMs in patients with cognitive impairment.

Study 2 Overview (Gedde et al)

Objective: To examine the effect of a deprescribing intervention (COSMOS) on medication use for nursing home residents.

Design: This was a randomized clinical trial.

Setting and participants: This trial was conducted in 67 units in 33 nursing homes in Norway. Participants were nursing home residents recruited from August 2014 to March 2015. Inclusion criteria included adults aged 65 years and older with at least 2 years of residency in nursing homes. Exclusion criteria included diagnosis of schizophrenia and a life expectancy of 6 months or less. Participants were followed for 4 months; participants were considered lost to follow-up if they died or moved from the nursing home unit. The analyses were per protocol and did not include those lost to follow-up or those who did not undergo a medication review in the intervention group. A total of 217 and 211 residents were included in the intervention and control groups, respectively.

Intervention: The intervention contained 5 components: communication and advance care planning, systematic pain management, medication reviews with collegial mentoring, organization of activities adjusted to needs and preferences, and safety. For medication review, the nursing home physician reviewed medications together with a nurse and study physicians who provided mentoring. The medication review involved a structured process that used assessment tools for behavioral and psychological symptoms of dementia (BPSD), activities of daily living (ADL), pain, cognitive status, well-being and quality of life, and clinical metrics of blood pressure, pulse, and body mass index. The study utilized the START/STOPP criteria¹ for medication use in addition to a list of medications with anticholinergic properties for the medication review. In addition, drug interactions were documented through a drug interaction database; the team also incorporated patient wishes and concerns in the medication reviews. The nursing home physician made final decisions on medications. For the control group, nursing home residents received usual care without this intervention.

Main outcome measures: The primary outcome of the study was the mean change in the number of prescribed psychotropic medications, both regularly scheduled and total medications (which also included on-demand drugs) received at 4 months when compared to baseline. Psychotropic medications included antipsychotics, anxiolytics, hypnotics or sedatives, antidepressants, and antidementia drugs. Secondary outcomes included mean changes in BPSD using the Neuropsychiatric Inventory-Nursing home version (NPI-NH) and the Cornell Scale for Depression for Dementia (CSDD) and ADL using the Physical Self Maintenance Scale (PSMS).

Main results: In both the intervention and control groups, 76% of participants were women, and mean age was 86.3 years (SD, 7.95) in the intervention group and 86.6 years (SD, 7.21) in the control group. At baseline, the mean number of total medications was 10.9 (SD, 4.6) in the intervention group and 10.9 (SD, 4.7) in the control group, and the mean number of psychotropic medications was 2.2 (SD, 1.6) and 2.2 (SD, 1.7) in the intervention and control groups, respectively. At 4 months, the mean change from baseline of total psychotropic medications was –0.34 in the intervention group and 0.01 in the control group (P < .001), and the mean change of regularly scheduled psychotropic medications was –0.21 in the intervention group and 0.02 in the control group (P < .001). Measures of BPSD and depression did not differ between intervention and control groups, and ADL showed a small improvement in the intervention group.

Conclusion: This intervention reduced the use of psychotropic medications in nursing home residents without worsening BPSD or depression and may have yielded improvements in ADL.

Commentary

Polypharmacy is common among older adults, as many of them have multiple chronic conditions and often take multiple medications for managing them. Polypharmacy increases the risk of drug interactions and adverse effects from medications; older adults who are frail and/or who have cognitive impairment are especially at risk. Reducing medication use, especially medications likely to cause adverse effects such as those with anticholinergic properties, has the potential to yield beneficial effects while reducing the burden of taking medications. A large randomized trial found that a pharmacist-led education intervention can be effective in reducing PIM use in community-dwelling older adults,² and that targeting patient motivation and capacity to deprescribe could be effective.³ This study by Bayliss and colleagues (Study 1), however, fell short of the effects seen in the earlier D-PRESCRIBE trial. One of the reasons for these findings may be that the clinician portion of the intervention was less intensive than that used in the earlier trial; specifically, in the present study, clinicians were not provided with or expected to utilize tools for structured medication review or deprescribing. Although the intervention primes the patient and family for discussions around deprescribing through the use of a brochure and questionnaire, the clinician portion of the intervention was less structured. Another example of an effective intervention that provided a more structured deprescribing intervention beyond education of clinicians utilized electronic decision-support to assist with deprescribing.⁴

The findings from the Gedde et al study (Study 2) are comparable to those of prior studies in the nursing home population,⁵ where participants are likely to take a large number of medications, including psychotropic medications, and are more likely to be frail. However, Gedde and colleagues employed a bundled intervention⁶ that included other components besides medication review, and thus it is unclear whether the effect on ADL can be attributed to the deprescribing of medications alone. Gedde et al’s finding that deprescribing can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression is an important contribution to our knowledge about polypharmacy and deprescribing in older patients. Thus, nursing home residents, their families, and clinicians could expect that the deprescribing of psychotropic medications does not lead to worsening symptoms. Of note, the clinician portion of the intervention in the Gedde et al study was quite structured, and this structure may have contributed to the observed effects.

Applications for Clinical Practice and System Implementation

Both studies add to the literature on deprescribing and may offer options for researchers and clinicians who are considering potential components of an effective deprescribing intervention. Patient activation for deprescribing via the methods used in these 2 studies may help to prime patients for conversations about deprescribing; however, as shown by the Bayliss et al study, a more structured approach to clinical encounters may be needed when deprescribing, such as the use of tools in the electronic health record, in order to reduce the use of medication deemed unnecessary or potentially harmful. Further studies should examine the effect of deprescribing on medication use, but perhaps even more importantly, how deprescribing impacts patient outcomes both in terms of risks and benefits.

Practice Points

• A more structured approach to clinical encounters (eg, the use of tools in the electronic health record) may be needed when deprescribing unnecessary or potentially harmful medications in older patients in community settings.
• In the nursing home setting, structured deprescribing intervention can reduce the use of psychotropic medications while not leading to differences in behavioral and psychologic symptoms or depression.

–William W. Hung, MD, MPH