Physicians and researchers at Johns Hopkins All Children’s Hospital have been working on a project since 2016 to improve the way the anesthesia team documents how controlled substances are used in the operating room. Their work, recently published in the journal Applied Clinical Informatics, is now being used in training materials for anesthesiologists across the country.

The Drug Enforcement Administration maintains a list of controlled substances that hospitals must report on. The list includes drugs such as fentanyl, morphine and methadone. When a physician gives a patient a dosage of one of these drugs, it is noted in the patient’s electronic medical record. The physician also has to note it in a separate medication management system. Because the systems don’t interface with each other, the pharmacy team reconciles the two systems to make sure they match.

A mismatch or discrepancy between the two systems typically points to human error. But because of the risk of diversion, which involves people with access to these medications stealing them for their own personal use or to sell illegally, hospitals are required to account for every milligram.

The rate of error in documenting the use of these drugs nationally is between 10-15%, mostly based on rates reported from adult hospitals. The rate at Johns Hopkins All Children’s was about 3% when the team began its work in June 2016, and routine monitoring has shown that diversion is not an issue.

This meant that the hospital’s team was seeing a little over 40 documentation mistakes due to human error each month out of 1,000 to 1,300 cases. Each mistake had to be identified and fixed individually. Now, after revising the process to cut down on the number of errors, the rate of error is less than 1%, which equates to about seven to 10 each month.

“The numbers were minimal” to start with, says Luis Ahumada, Ph.D., director of predictive analytics in the hospital’s Health Informatics core. “But part of taking good care of our patients is having good documentation of what we do.”

The director of the pharmacy program brought the rate of errors to the attention of Jenny Dolan, M.D., medical director for the Department of Anesthesia and Pain Management, who brought the issue to the entire anesthesia group so they could work together to fix it.

Dolan gave this scenario as an example of the kinds of discrepancies the team was seeing:

A physician receives a 10-microgram syringe from the pharmacy and plans to administer five micrograms to the patient in the OR. The physician moves the patient to the pediatric intensive care unit (PICU) where it’s determined she needs the additional five micrograms. The physician notes the full amount the patient received — 10 micrograms — in her electronic medical record. But the physician forgot to note this in the other medication management system and “tell” the pharmacy that all 10 micrograms were used.

“That’s only 10 micrograms, but those add up,” Ahumada says.

The first step was to establish a regular meeting where the team reported on progress. Penny Lyman, RcPhT, regulatory pharmacy technician, reconciled the reports daily to identify errors, a process requiring diligence and hard work, Dolan says. The team also took the small but important step of adding signs to certain spots in the OR to serve as visual reminders to the care teams to document the controlled substances they administered properly in all the right places.

The team used visual analytics throughout the process. Visual analytics allow researchers to take large amounts of data and depict it in a way that is easier for people to digest. In this case, the team used rotating graphics that charted each physician’s progress on a large television screen in the employee lounge to encourage everyone to improve and keep the issue top of mind.

“We would never have been able to make as much progress as we have without that,” says Allison Fernandez, M.D., pediatric anesthesiologist. “It makes it easy to see and make adjustments using this dashboard, and it facilitates looking at data in a very different way.”

“People could see how they were performing, and how the system was changing through time. That was crucial,” Dolan says. “It makes you compare yourself to the rest of the people and it also gave the good news that we were doing better.”

“As clinicians, we’re always looking for ways to improve our practice,” says Anna Varughese, M.D., director of perioperative quality and safety in the Department of Anesthesia. “Process improvements are important so we ultimately are doing the best for our patients.”

However, “it’s difficult to improve performance by setting an arbitrary goal, i.e., we want to reduce errors by this much. It’s an abstract thought to most providers,” she says. “But when you can obtain and display both aggregate and individual performance data in real-time, as well as provide immediate feedback to clinicians on their everyday performance, it’s much easier to improve.”

The team determined the national error rate for hospitals was 10-15% by pooling together the various error rates reported in academic literature on the topic. There was no true agreed upon national standard to measure the hospital against, but the team focused on improving its 3% error rate. The process improvements also help to ensure that patient families aren’t overcharged for medications because of a reporting error in the pharmacy’s records.

The team’s work was published in the journal Applied Clinical Informatics. The Johns Hopkins All Children’s team involved in the publication included Dolan; Hannah Lonsdale, visiting scientist; Ahumada; Amish Patel, M.D., pediatric anesthesiologist; Jibin Samuel, M.D., pediatric anesthesiologist in pain management; Ali Jalali, Ph.D., data scientist in Health Informatics; JoAnn DeRosa, clinical research coordinator; Mohamed Rehman, M.D., chair of the Department of Anesthesia and Pain Medicine; Varughese and Fernandez.

The American Board of Anesthesiology is using the study as part of its redesigned Maintenance of Certification in Anesthesiology™ program, which provides continuing education opportunities for anesthesiologists, to show how the use of visual analytics can impact quality improvement projects. As the team continues using visual analytics to improve this documentation process, they are also exploring other ways that visual analytics can help with quality improvement projects in other areas. Dolan will also present the study at American Society of Anesthesia’s Wake Up Safe conference in October.