Latest Overview of RAG: Review of 15 Classic RAG Frameworks (Part 2)

Source: Deep Graph Learning and Large Model LLM

This article is approximately 3500 words long and is recommended for a 9-minute read. It delves into the development of Retrieval-Augmented Generation (RAG), from basic concepts to the latest technologies.

4. Overview of Existing RAG Frameworks

Agent-Based RAG

A new agent-based Retrieval-Augmented Generation (RAG) framework employs a hierarchical multi-agent structure, where sub-agents fine-tune small pre-trained language models (SLMs) for specific time-series tasks. The main agent assigns tasks to these sub-agents and retrieves relevant prompts from a shared knowledge base. This modular multi-agent approach achieves high performance, demonstrating flexibility and efficiency in time-series analysis compared to task-specific methods.

RULE

RULE is a multi-modal RAG framework designed to enhance the factual accuracy of medical visual-language models (Med-LVLM) by introducing calibration selection strategies to control factual risks and developing preference optimization strategies to balance the model’s intrinsic knowledge with the retrieval context, demonstrating its effectiveness in improving the factual accuracy of Med-LVLM systems.

METRAG

METRAG, a multi-level, thoughts-enhanced retrieval-augmented generation framework, combines document similarity and utility to enhance performance. It includes a task-adaptive summarizer to produce distilled content summaries. By leveraging multiple reflections during these stages, LLM generates knowledge-enhanced content, showcasing superior performance compared to traditional methods in knowledge-intensive tasks.

RAFT (Retrieval Augmented Fine-Tuning)

Disruptive documents are a key feature of Retrieval Augmented Fine-Tuning (RAFT) (Zhang et al., 2024), training the model to discern using irrelevant, disruptive documents while directly citing relevant sources. Combined with chain-of-thought reasoning, it enhances the model’s reasoning capabilities. RAFT shows consistent performance improvements in specific domain RAG tasks, including PubMed, HotpotQA, and Gorilla datasets, as a post-training enhancement for LLMs.

FILCO

FILCO aims to improve the contextual quality provided by generative models in open-domain question answering and fact verification tasks, addressing the issues of over-reliance or under-reliance on retrieved paragraphs, which may lead to hallucination problems in generated outputs. This method identifies useful contexts through lexical and information-theoretic approaches and refines the retrieved context during testing by training context filtering models to enhance contextual quality.

Self-RAG

Reflective tokens are a key attribute of Self-Reflective Retrieval-Augmented Generation (Self-RAG) (Asai et al., 2023), which improves the factual accuracy of large language models (LLMs) by combining retrieval and self-reflection. Unlike traditional methods, Self-RAG adaptively retrieves relevant paragraphs and uses reflective tokens to evaluate and refine its responses, allowing the model to adjust its behavior according to specific task requirements, demonstrating superior performance in open-domain question answering, reasoning, fact verification, and long-form generation tasks. The intelligence and effectiveness of RAG largely depend on retrieval quality, and a better understanding of metadata in the knowledge base will enhance the effectiveness of RAG systems.

MK Summary

A data-centric Retrieval-Augmented Generation (RAG) workflow that transcends the traditional retrieve-read paradigm, adopting a prepare-rewrite-retrieve-read framework to enhance LLMs by integrating contextually relevant, time-critical, or domain-specific information. Its innovations include generating metadata, synthesizing questions and answers (QA), and introducing meta-knowledge summaries (MK summaries) of document clusters.

CommunityKG-RAG

CommunityKG-RAG is a zero-shot framework that integrates community structure from knowledge graphs (KGs) into retrieval-augmented generation (RAG) systems. By leveraging multi-hop connections within KGs, it improves the accuracy and contextual relevance of fact-checking, surpassing traditional methods that do not require additional domain-specific training.

RAPTOR

RAPTOR introduces a hierarchical approach to enhance retrieval-augmented language models, addressing the limitation of traditional methods that only retrieve short, contiguous text blocks. RAPTOR retrieves information by recursively embedding, clustering, and summarizing text, forming a summary tree to retrieve information at different levels of abstraction. Experiments show that RAPTOR demonstrates superior performance in question-answering tasks requiring complex reasoning. When paired with GPT-4, RAPTOR’s accuracy improved by 20% on the QuALITY benchmark.

4.1 Long Context-Based RAG Frameworks

Recently launched large language models (LLMs) supporting long contexts, such as Gemini-1.5 and GPT-4, have significantly enhanced RAG performance.

Self-Route

Self-Route dynamically allocates queries to RAG or LC through model introspection, optimizing computational costs and performance. It provides profound insights into the best applications of RAG and LC when handling long-context tasks.

SFR-RAG

SFR-RAG is a compact and efficient RAG model designed to enhance LLMs’ integration of external contextual information while reducing hallucination phenomena.

LA-RAG

LA-RAG is a new RAG paradigm aimed at enhancing automatic speech recognition (ASR) capabilities in LLMs. Its highlight is the ability to utilize fine-grained token-level speech data storage and speech-to-speech retrieval mechanisms to improve ASR accuracy through contextual learning in LLMs.

HyPA-RAG

LLMs face challenges due to outdated knowledge and hallucinations in the context of AI law and policy. HyPA-RAG is a hybrid parameter adaptive retrieval-augmented generation system that improves accuracy through adaptive parameter adjustments and hybrid retrieval strategies. In tests on NYC Local Law 144, HyPA-RAG demonstrated higher correctness and contextual accuracy, effectively addressing the complexities of legal texts.

MemoRAG

MemoRAG introduces a new RAG paradigm that overcomes the limitations of traditional RAG systems when handling vague or unstructured knowledge. MemoRAG’s dual-system architecture utilizes lightweight long-distance LLMs to generate draft answers and guide retrieval tools, while a more powerful LLM is responsible for refining the final output. This framework is optimized for better clue extraction and memory capacity, significantly outperforming traditional RAG models in both complex and simple tasks.

NLLB-E5

NLLB-E5 introduces a scalable multilingual retrieval model that addresses the challenges of supporting multiple languages, especially low-resource languages like Hindi. Utilizing the NLLB encoder and E5 multilingual retriever’s distillation method, NLLB-E5 enables zero-shot retrieval across languages without multilingual training data. Evaluations on benchmarks like Hindi-BEIR show its robust performance, highlighting task-specific challenges and promoting global inclusivity in multilingual information retrieval.

5. Challenges and Limitations of RAG

• Scalability and Efficiency:One major challenge of RAG is its scalability. Given that the retrieval component relies on external databases, efficient retrieval algorithms are needed to cope with large and increasingly growing datasets. High computational and memory demands also make it difficult to deploy RAG models in real-time or resource-constrained environments.
• Retrieval Quality and Relevance:Ensuring the quality and relevance of retrieved documents is a significant issue. Retrieval models may sometimes return irrelevant or outdated information, which can reduce the accuracy of generated content. Particularly in long-form content generation, improving retrieval accuracy remains a hot research topic.
• Bias and Fairness:Like other machine learning models, RAG systems may exhibit bias due to biases in the retrieval datasets. Retrieval-based models may amplify harmful biases present in the retrieved knowledge, leading to biased outputs. Developing bias mitigation techniques for retrieval and generation is an ongoing challenge.
• Coherence:RAG models often struggle to integrate retrieved knowledge into coherent, contextually relevant text. The linkage between retrieved content and generated model outputs is not always seamless, which can lead to inconsistencies or factual hallucinations in the final responses.

• Interpretability and Transparency:Like many AI systems, RAG models are often viewed as opaque black boxes.

6. Future Directions

6.1 Strengthening Multimodal Integration

Integrating text, image, audio, and video data in RAG models requires focusing on enhancing multimodal fusion technologies to enable seamless interactions between different data types, including:

• Developing more advanced methods to align and synthesize cross-modal information.
• More innovation is needed to enhance the coherence and situational adaptability of multimodal outputs.

• Enhancing the ability of RAG systems to retrieve relevant information across different modalities. For example, combining text-based queries with image or video content retrieval can enhance applications such as visual question answering and multimedia search.

6.2 Scalability and Efficiency

As RAG models are deployed in broader large-scale applications, scalability becomes crucial. Research should focus on developing efficient methods for scaling retrieval and generation processes without sacrificing performance. Distributed computing and efficient indexing techniques are vital for handling large datasets. Improving the efficiency of RAG models requires optimizing retrieval and generation components to reduce computational resources and latency.

6.3 Personalization and Adaptability

Future RAG models should focus on personalizing the retrieval process based on individual user preferences and contexts. This includes developing techniques to adjust retrieval strategies based on user history, behavior, and preferences. Enhancing the contextual adaptability of RAG models is crucial for improving the relevance of generated responses by deeply understanding the context and sentiment of queries and document libraries. Research should explore methods for dynamically adjusting retrieval and generation processes based on interactive contexts, including integrating user feedback and contextual cues into the RAG workflow.

6.4 Ethical and Privacy Considerations

Addressing bias is a key area of future research, particularly concerning biases in RAG models. As RAG systems are deployed in diverse applications, ensuring fairness and reducing biases in retrieved and generated content is essential. Future RAG research should focus on privacy-preserving techniques to protect sensitive information during retrieval and generation processes, including developing secure data processing methods and privacy-aware retrieval strategies. The interpretability of models is also a key area for ongoing improvement in RAG research.

6.5 Cross-Language and Low-Resource Language Support

Expanding RAG technologies to support multilingual capabilities, especially for low-resource languages, is a promising direction for development.

Efforts should be made to enhance cross-language retrieval and generation capabilities, ensuring accurate and relevant results across different languages. Improving RAG models’ effective support for low-resource languages requires developing methods for content retrieval and generation under limited training data. Research should focus on transfer learning and data augmentation techniques to improve performance in edge languages.

6.6 Advanced Retrieval Mechanisms

Future RAG research should explore dynamic retrieval mechanisms that adapt to changing query patterns and content needs. This includes building models that can dynamically adjust retrieval strategies based on new information and user needs.

Researching hybrid retrieval methods that combine dense and sparse retrieval strategies holds promise for enhancing RAG system effectiveness. Research should focus on how to integrate diverse retrieval methods to adapt to various tasks and achieve optimal performance.

6.7 Integration with Emerging Technologies

Combining RAG models with brain-computer interfaces (BCIs) could open new applications in human-computer interaction and assistive technologies. Research should explore how RAG systems can leverage BCI data to enhance user experience and generate context-aware responses. The integration of RAG with augmented reality (AR) and virtual reality (VR) technologies presents opportunities to create immersive interactive experiences. Future research should investigate how RAG models can be utilized to enhance AR and VR applications by providing contextually relevant information and interactions to enhance user experience.

• Original paper: https://arxiv.org/abs/2410.12837