The combination of AI and smart healthcare is an inevitable trend for future development. In recent years, the National Health Commission has promoted smart healthcare and AI technologies, such as intelligent triage, pre-consultation, and diagnostic assistance, to improve the efficiency of medical services and the accuracy of diagnoses, enhancing the patient experience.
Quanzhentong is a company focused on medical SaaS and artificial intelligence, established in 2016. Its AI medical service agent based on large models covers patient services, intelligent cases, decision support, and medical education, providing a “foundation-model, application” one-stop solution for medical clients, aiming to enhance the efficiency and quality of medical services. As of now, Quanzhentong’s products and solutions have been successfully applied in 16 provinces, including over 50 large and medium-sized hospitals, more than 100 district health bureaus, and over 15,000 medical institutions.
01.
Milvus Supports RAG-Based Medical Knowledge Retrieval
Milvus, as the vector search engine and storage for the entire RAG application, combines general medical knowledge with proprietary hospital medical knowledge, serving RAG applications in medical knowledge retrieval, Q&A scenarios, and AI-assisted diagnosis.
1.1 Knowledge Q&A in Medical Scenarios
For example, the “patient service AI” application primarily aims to alleviate the suffering of patients, providing AI triage, AI pre-consultation, and medical answers.
In the case of hospital triage, there are multiple campuses, and patient inquiries may involve queries about general hospital knowledge (such as inquiry about the hospital admission process or whether to fast before a B-ultrasound) and proprietary campus knowledge (such as the reason for a B-ultrasound: digestive system issues or gynecological examination?). This requires joint retrieval from both the general hospital and campus knowledge bases to improve query efficiency. Milvus establishes different partitions for general knowledge and proprietary campus knowledge. When users consult questions, the triage system searches both partitions simultaneously and merges the results to provide feedback to the large model for result generation.
This upgrades the traditional triage based on knowledge graphs to an AI-driven precise guiding inquiry based on the patient’s specific description, combined with age, gender, and other information, to accurately guide patients in questioning. AI analyzes the diagnosis and hospital department information to recommend the appropriate department for consultation.
For instance, Quanzhentong has provided a full-scenario application of triage + pre-consultation + outpatient + inpatient + surgery for a certain city’s First People’s Hospital, becoming a versatile assistant for doctors.
1.2 AI-Assisted Diagnosis
Clinical decision-making relies on a large amount of medical knowledge and historical cases. Different hospitals have unique specialties, treatment plans, and corresponding specifications based on their type and audience, such as department-specific coding, templates, and preferred patient case data, thus requiring flexible AI-assisted diagnostic solutions. Quanzhentong AI embeds massive amounts of unstructured medical literature, case libraries, clinical guidelines, etc., from various hospitals/campuses into Milvus. During outpatient consultations, doctors only need to start “Quanzhentong AI” with one click, and the assisted diagnostic process is as follows:
-
AI automatically records doctor-patient conversations, embedding the voice for semantic extraction;
-
Milvus retrieves the conversational semantic information as a prompt from massive medical guidance data feature data;
-
Through RAG technology, relevant information related to the current patient situation is quickly found to provide real-time reminders and guidance for doctors.
RAG-based AI-assisted diagnosis has revolutionized the doctor-patient communication method, significantly reducing error rates and ensuring the comprehensiveness, accuracy, and timeliness of inquiry decisions.
1.3 Application of Medical Imaging
Medical imaging is a massive business in medical work. For example, a certain tertiary hospital in Hangzhou needs to handle millions of ultrasound examinations each year. During the examination process, ultrasound doctors need to use both hands for operation, making it impossible to write reports. To solve this problem, Quanzhentong AI proposed a question-answer model based on RAG technology and SFT (Supervised Fine-Tuning) technology to improve work efficiency in this regard. For instance, it provides an efficient and accurate end-to-end solution for ultrasound image recognition and report generation, realizing the transition of AI from serving patients to serving doctors.
RAG technology retrieves relevant information from external knowledge bases and provides it as context to the language model, enhancing the model’s ability to generate text. SFT technology further improves the model’s adaptability and accuracy through retraining and reinforcement learning on a small amount of labeled data.
Milvus plays a crucial role in this process, mainly responsible for storing and retrieving high-dimensional data, such as vector representations of ultrasound images, enabling RAG technology to quickly retrieve the most relevant information from a large amount of data. Additionally, Milvus supports RAG technology’s ability to handle complex queries and generate information-rich responses, which is particularly important for ultrasound image recognition in the medical field, as it requires accurate understanding and generation of natural language instructions.
This method can free up hands for writing ultrasound reports, improving work efficiency. Moreover, this technology can also be applied in small hospitals, potentially increasing overall efficiency by more than 100%.
02.
Upgrading Medical Models to Enhance Business Efficiency
The entire medical model has transitioned from traditional knowledge graphs or expert system rule-based methods to new approaches utilizing large models. This transformation directly leads to an enhancement in overall medical efficiency. Taking AI triage as an example, Quanzhentong’s chief model architect Liu Rui stated that traditional AI triage required more than two people about a month to complete system setup, whereas now the implementation time has been shortened to about a week. In terms of manpower and resource investment, the vector engine plays a crucial role.
03.
Future Exploration: End-to-End Medical Large Models with Multi-Modal Capabilities
In the future, Quanzhentong’s product goal is to empower Quanzhentong AI with multi-modal technology to become an all-around assistant for doctors and patients.
In the past, hospitals’ ultrasound models mainly employed traditional image detection and classification technologies, focusing on recognizing image content, lesion types, and their characteristics. Moving forward, Quanzhentong plans to use case data to train the models. To achieve this goal, medical units will need to store large amounts of case data in vector databases for more effective management and retrieval of this data.
04.
Interlude: Milvus Has Always Been the Preferred Choice for Vector Retrieval Scenarios
When asked why Milvus was chosen, Liu Rui recalled that he had encountered Milvus in his previous work experience and researched various mainstream vector retrieval platforms, finding that Milvus precisely met the business needs at that time. The most important reasons for this choice include:
-
Milvus is excellent, known for its high performance, low latency, high QPS, and strong scalability;
-
It also supports both vector retrieval and scalar filtering, simplifying query operations while enhancing result accuracy;
-
Support for partitioning, as many e-commerce business scenarios require cross-database queries, using partition queries to perform similar cross-database queries while maintaining performance. For example, when searching for brand terms, the process is cross-category; searching within a single category is insufficient, which corresponds to the same scenario in medical Q&A.
Later, when multiple businesses involved text search, image search, and RAG scenarios, Milvus became Liu Rui’s unequivocal choice.
Recommended Reading
