Empowering Clinical Medicine with Intelligent Voice Recognition Technology

“Medical records document the occurrence, development, outcome, examination, diagnosis, and treatment processes of diseases. They summarize clinical practice and serve as a basis for exploring disease patterns. The quality of medical records represents the quality of medical care, and standards must never be lowered!” “The inpatient electronic medical record is templated, and a few doctors choose to ‘copy’ and ‘paste’ to save time, losing the personalized descriptions of patients. Modern doctors work under greater intensity and faster pace, but the quality of medical records must not be sacrificed!”

Communication meeting of the medical record quality control expert group

In August 2014, a medical record quality control meeting held at Peking Union Medical College Hospital was particularly lively. Senior experts reiterated the importance of medical record quality, and management team members from the medical department, information management department, and medical record department worked together to find ways to relieve the burden on doctors. “So can Peking Union develop its own voice input system?”

To assess the application value of the voice input system, Peking Union Medical College Hospital immediately conducted extensive research. Data shows that the workload of electronic text input in Chinese hospitals is large, with over 40% of doctors spending about 4 hours a day on text input in front of computers, and over half of doctors spending 40% of their total work time on text input. There is a high demand among doctors to improve the efficiency of medical record entries. Voice recognition technology has been used in the medical field in Europe and America for nearly 10 years, with the application rate of voice recognition input in American clinics reaching 10% to 20%. Language recognition technology software and hardware products have also appeared in the Chinese market. Although the application of voice recognition input has great prospects, there are many difficulties. The 415 untoned syllables in Chinese constitute more than 7,000 commonly used Chinese characters, and homophones are common; there are many dialects and regional accents; there are many medical terminologies; and the working environment of doctors is noisy, often with multiple people speaking at the same time, which increases the difficulty of voice recognition. After comprehensive analysis of the pros and cons, the innovative people at Peking Union decided to bravely face the challenges and began developing the voice input system in September 2014.

To ensure that the system understands doctors’ speech, the information management department first sorted through the complete medical record data of each department, compiling more than 30GB of medical text data to train and learn the system on “Peking Union’s proprietary corpus.” After deep custom development on medical character vocabulary, common diseases, drug names, and operational steps, a high-standard, high-recognition effect, and doctor-friendly voice recognition model was obtained. Doctors in different specialties have personalized language characteristics, and exclusive development for different specialties further improves recognition accuracy. Doctors can also add commonly used vocabulary according to their language habits in the system.

In practical scenarios, doctors speak into the microphone on the desk: “The patient is on the 10th day of hospitalization, blood pressure 120/80 mmHg, weight loss 2.5 kg, white blood cell count 6 times 10 to the 9th power per liter, cardiopulmonary examination is negative.” The system displays: “The patient is on the tenth day of hospitalization, blood pressure 120/80 mmHg, weight loss 2.5 kg, blood routine: WBC 6×10⁹ /L, cardiopulmonary examination is (-).” After more than half a year of development, in March 2015, the voice input system was first piloted in the basic surgery and orthopedics wards, immediately receiving positive feedback from clinical doctors. The USB microphone is plug-and-play, requiring no hardware drivers, and after installing the voice input software, it can directly interface with clinical information systems like HIS.

In August 2015, Peking Union Medical College Hospital, together with its development partners, applied for copyright of the medical voice input software. In March 2016, an analysis of 135 randomly sampled cases, totaling over 25,000 words, showed that the intelligent voice input system at Peking Union Medical College Hospital achieved an accuracy rate exceeding 95% in quiet environments, with doctors needing to manually correct only 2 times per 100 words. The voice input system was rapidly promoted from pilot wards to the Dongdan campus and then to the Xidan campus, going live in all wards in September 2016.

Through continuous “tailor-made” adjustments, voice assistants have entered more medical scenarios, making Peking Union Medical College Hospital the first Class A hospital in the country to scale the application of the intelligent voice input system.

In December 2017, the ultrasound voice assistant was trialed in the outpatient examination room, supporting insertion, modification, deletion, as well as conventional editing commands like cursor movement, line breaks, and undo. A single doctor can complete the entire process from calling a number, image collection, editing reports to printing reports using voice commands. The application uses wake-free technology, allowing voice input to remain active at all times. During examinations, the system intelligently filters the conversations between doctors and patients, only recording and executing the content related to reports that the doctor dictates. The use of mobile wireless microphones instead of goose-neck microphones in wards and outpatient settings completely frees ultrasound doctors’ hands, allowing them to focus on operating ultrasound equipment.

Ultrasound doctors using voice recognition technology to achieve full-process voice command control

Starting in January 2018, Peking Union Medical College Hospital began trialing the voice input system in outpatient settings to align with the launch of the outpatient electronic medical record system. Once the electronic medical record system went live, patients no longer needed to carry the small blue outpatient medical record.

Voice assistant functionality on the official Peking Union App

In July 2018, the patient app voice assistant was launched. At the bottom of the homepage of the official Peking Union Medical College Hospital app, clicking the voice icon and holding down to speak allows users to query doctors, register, navigate, etc. The app voice assistant also provides a simple triage function, guiding patients to register for the corresponding specialty by stating their symptoms.

To understand the actual usage situation and further optimize and improve the system, in March 2016, the information management department distributed and collected 102 surveys from participating doctors via WeChat work groups. Doctors generally expressed that voice recognition technology effectively improved work efficiency, allowing more time for patient care. 90% of doctors believed the voice input system helped them improve work efficiency, and over 50% of doctors felt they could save about 1 hour of time each day. The average frequency of using voice input is 1 to 2 times a day.

Further data analysis revealed that departments with more complicated and serious cases had a higher frequency of voice input usage and better results. This is related to the special disease source characteristics of Peking Union Medical College Hospital. As a national center for the diagnosis and treatment of difficult and severe cases, Peking Union receives a large number of “old” patients who have visited multiple hospitals without effective treatment, often bringing thick medical records. The “efficient weapon” for summarizing and inputting these medical records is the voice input system.

The more frequently used wards are shown in redder colors and larger fonts in the image

Surveys found that the main reason doctors dislike using voice input is that they are used to thinking and organizing language while typing on the keyboard, rather than organizing a large segment of language and inputting it all at once. Some doctors also mentioned that they were often interrupted by patients asking about their conditions while using voice input. Regardless, Peking Union Medical College Hospital bravely paved the way, providing valuable experience for the clinical application of intelligent voice recognition technology.

– – – End – – –

Text | Fu Tanping, Chen Jieqing, Zhao Congpu

Editor | Guo Jing

Images | Provided by the Information Management Department