Abstract:

The transition from traditional RAG to Graph RAG enhances large language models by integrating knowledge graphs, enabling them to provide more detailed and accurate responses to complex queries. The effectiveness of Graph RAG also depends on the quality and breadth of the underlying knowledge graph and the engineering aspects of RAG.

Main Points:

– Graph RAG improves LLM responses by utilizing knowledge graphs for enriched context.

– It addresses the limitations of traditional RAG, enhancing understanding of complex queries.

– Complex retrieval and indexing methods increase the accuracy and speed of RAG systems.

– Future integrations may lead to dynamic updates, improved computational efficiency, and broader knowledge transfer.

– The impact of Graph RAG spans areas such as natural language processing, AI ethics, and AI innovation.

– Knowledge graphs play a crucial role in providing structured, context-relevant data to LLMs.

– The effectiveness of Graph RAG depends on the quality and breadth of the underlying knowledge graph and the engineering aspects of RAG.

Introduction

We know that large language models (LLMs) operate on fixed datasets, with their knowledge stopping at the last training update.

Regular users of ChatGPT may recognize this well-known limitation: “Training data cut-off is September 2021.” This limitation can lead to inaccurate or outdated responses, as these models may “hallucinate” information.

Updating them with new information or enhancing contextual understanding without retraining or fine-tuning can be challenging and resource-intensive.

Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation (or RAG for short) was introduced as a technique to improve large language models (LLMs) by merging information from external reliable knowledge bases.

The principle behind RAG is simple; when asked to answer a question, the LLM does not solely rely on known content. Instead, it first searches for relevant information from specific knowledge sources.

This approach ensures that the generated output references a wealth of context-rich data, enhancing it with current and relevant information.

RAG primarily operates through a two-phase process: Retrieval and Content Generation.

Retrieval Phase

In the retrieval phase, the algorithm locates and collects information fragments relevant to the user’s prompt or query.

For example, if you are looking for a recipe for Fujian fried noodles, your prompt might be “What are the ingredients for Fujian fried noodles?”

The system identifies documents with content related to the query’s semantics and uses a similarity measure (often cosine similarity) to calculate relevance by assessing the similarity between their vectors.

After summarizing external knowledge, the system appends it to the user’s prompt content and sends it as enriched input to the language model.

Content Generation Phase

In the subsequent generation phase, the LLM combines this enriched prompt with its own training data representation to generate a response tailored to the user’s query. This response provides a blend of personalized and verifiable information suitable for applications like chatbots.

From Traditional RAG to Graph RAG - When Large Models Meet Knowledge Graphs — Retrieval Augmented Generation (RAG)

Why RAG Matters

In today’s technological landscape, LLMs power many natural language applications, enabling them to understand or generate human-like text. Despite being very powerful, these models can sometimes underperform.

Sometimes, they can be overly confident in generating incorrect responses, making it easy for people to be swayed by their compelling arguments.

RAG aims to mitigate these issues by guiding LLMs to obtain information from trustworthy sources, thus maintaining the relevance and accuracy of model outputs.

Limitations of RAG

Like everything in life, traditional RAG methods come with complexities and challenges.

While groundbreaking in enhancing LLM capabilities, RAG also has certain limitations that may affect its effectiveness and applicability.

One major challenge involves the accuracy of retrieved information and the multi-source heterogeneity of data sources.

For RAG to work, it often relies on multiple external sources, which may vary in format, standards, and reliability levels (e.g., PDF, Flat files, Markdown, CSV, Web content).

Implementing RAG also faces issues with ambiguous queries or the need for deep contextual understanding.

These problems, as inherent issues of technical design, primarily stem from the retrieval process, which sometimes overlooks nuances critical for precise responses.

Enhancements to RAG

Enhancing the retrieval accuracy and efficiency of RAG systems is an ongoing research direction in the fields of natural language processing and machine learning. Several strategies can be adopted to achieve these improvements, but I want to highlight two notable enhancements that can be realized with today’s technology.

Implementing more sophisticated retrieval algorithms to better understand the semantics of queries can improve the relevance of retrieved documents.
Efficiently indexing the knowledge base to speed up the retrieval process without sacrificing the quality of results.

This leads us to our next topic…

Graph RAG: RAG x Knowledge Graphs

Graph RAG builds on the concept of RAG while leveraging knowledge graphs (KGs).

This innovative approach, proposed by NebulaGraph, Neo4J, Microsoft, etc., changes the way LLMs interpret and respond to queries by integrating graph databases.

Graph RAG runs by integrating structured data from KGs into the processing of LLMs, providing a more detailed and sufficient basis for the model’s responses. KGs are structured representations of real-world entities and their relationships, consisting of nodes and edges, where nodes represent entities such as people, places, objects, or concepts, and edges represent the relationships or connections between these entities.

This structure greatly enhances the ability of LLMs to generate intelligent responses, allowing models to access precise and context-relevant data.

The innovation of Graph RAG lies in integrating graph databases with LLMs, enriching the model’s context before generating responses.

Some popular graph database products include Ontotext, NebulaGraph, and Neo4J.

What Makes Graph RAG Significant

As LLMs continue to grow in precision and capability, Graph RAG has the potential to significantly impact the field of artificial intelligence.

Here are my visions for the development of this integration:

1. Future LLMs are expected to demonstrate improved understanding of complex queries and reasoning capabilities.

Graph RAG can leverage these advancements to provide more precise and contextually relevant answers.

Structured knowledge derived from knowledge graphs, combined with more complex LLMs, may lead to breakthroughs in AI’s ability to understand abstract concepts, reason, and produce nuanced responses.

2. As LLMs seem to progress uncontrollably, their integration with knowledge graphs may become more dynamic and seamless.

This may involve real-time updates of knowledge graphs based on global events or discoveries.

LLMs can play a role in automatically enhancing and updating knowledge graphs by integrating new information collected from user interactions or other data sources.

Techniques like Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning from AI Feedback (RLAIF) can further help align models with human preferences and adhere to the HHH (helpful, honest, and harmless) principles.

3. With NVIDIA’s efforts to democratize AI computing, future advancements achieved through LLMs and Graph RAG will focus on increasing computational efficiency and scalability.

This shift will enable Graph RAG to be used in a wider range of applications, including those requiring real-time responses or operating in resource-constrained environments.

4. Future LLMs are expected to have broader and deeper knowledge across multiple fields.Graph RAG can facilitate knowledge transfer across different domains, enabling insights or solutions that utilize information from various fields.

For example, applying findings from cognitive science may lead to the development of more natural human-computer interaction models, or combining cybersecurity with psychology may enhance the effectiveness of security measures.

5. As Graph RAG technology advances, adopting standards like the Resource Description Framework (RDF) for knowledge graphs can improve interoperability between various systems.

This may mean that different implementations interact and collaborate, driving broader adoption and innovation.

Graph RAG Demo

For this demonstration, we will use product information from the Govtech developer portal as our knowledge base.

Setup

Start a local instance of Neo4j using Neo4j Desktop.

Use LangChain to connect to the Neo4j database locally. The good news is that LangChain has a template available for quick setup.

2. Extraction

Use prompt engineering and LLM to extract information, nodes, and their connections. Here’s an example of a prompt:

# Instructions for Creating Knowledge Graphs
## Overview
You are engineered for organising data into knowledge graphs.
- **Nodes**: Represent entities and ideas.
- The objective is to ensure the knowledge graph is straightforward and intelligible for broad use.

## Node Labeling
- **Uniformity**: Stick to simple labels for nodes. For instance, label any entity that is an organisation as "company", rather than using terms like "Facebook" or "Amazon".
- **Identifiers for Nodes**: Opt for textual or comprehensible identifiers over numerical ones.
  - **Permissible Node Labels**: If there are specific allowed node labels, list them here.
  - **Permissible Relationship Types**: If there are specific allowed relationship types, list them here.

## Managing Numerical Data and Dates
- Integrate numerical information directly as attributes of nodes.
- **Integrated Dates/Numbers**: Refrain from creating distinct nodes for dates or numbers, attaching them instead as attributes.
- **Format for Properties**: Use a key-value pairing format.
- **Avoiding Quotation Marks**: Do not use escaped quotes within property values.
- **Key Naming**: Adopt camelCase for naming keys, such as `dateTime`.

## Uniformity
- **Entity Uniformity**: Ensure consistent identification for entities across various mentions or references.
  
## Adherence to Guidelines
Strict adherence to these instructions is mandatory. Non-adherence will result in termination.

3. Graph Construction

Use CSVLoader and document splitting to process our documents.
Map the extracted information to graph nodes and relationships.
Process documents through our extraction pipeline and store the information in Neo4j.

The entire process takes about an hour, resulting in a final list of extracted node labels.

Unfortunately, not all node labels are useful or meet our needs in context.

{
  "identity": 1040,
  "labels": [
    "Feedbackstatus"
  ],
  "properties": {
    "id": "Feedback-Success",
    "message": "Sent. Thank you for the feedback!"
  },
  "elementId": "4:81cd2613-0f18-49c1-8134-761643e88b7a:1040"
},
{
  "identity": 1582,
  "labels": [
    "Feedbackstatus"
  ],
  "properties": {
    "id": "Feedbacksuccess",
    "status": "Sent. Thank you for the feedback!"
  },
  "elementId": "4:81cd2613-0f18-49c1-8134-761643e88b7a:1582"
},
{
  "identity": 1405,
  "labels": [
    "Header"
  ],
  "properties": {
    "id": "Modalcardhead",
    "class": "sgds-modal-card-head"
  },
  "elementId": "4:81cd2613-0f18-49c1-8134-761643e88b7a:1405"
},
{
  "identity": 1112,
  "labels": [
    "Feedbackindicator"
  ],
  "properties": {
    "id": "Feedbacksuccess",
    "title": "check",
    "message": "Sent. Thank you for the feedback!"
  },
  "elementId": "4:81cd2613-0f18-49c1-8134-761643e88b7a:1112"
}...

4. Evaluation and Improvement

We will specify which node labels the LLM should extract to refine our approach.

permissible_nodes_to_extract = [
    "Aisubfield",
    "Application",
    "Cloudservice",
    "Concept",
    "Digitalsolution",
    "Division",
    "Entity",
    "Feature",
    "Fundinginitiative",
    "Initiative",
    "Link",
    "Location",
    "Organization",
    "Person",
    "Platform",
    "Policy",
    "Program",
    "Resource",
    "Role",
    "Schema",
    "Service",
    "Standard",
    "Technology",
    "Technologyplatform",
    "Technologystack",
    "Webframework",
    "Webresource",
    "Website"
]

Explore our newly constructed knowledge graph using the Neo4j browser.

Continuously refine our extraction and graph construction methods.

After all the efforts, the magic finally begins! Consider the following query:

Ask: “Tell me more about AiSay, including the people who developed this product, as well as its benefits and applications. Also, please tell me how AiSay is unique.“

Basic RAG: “AiSay is an AI-driven document reader developed by the Singapore government. It can automatically detect, scan, and understand various types of documents, including handwritten notes, printed articles, multilingual texts, and structured/unstructured documents. Unlike traditional OCR systems, AiSay can provide structured responses for documents like images and PDFs. It leverages OCR, document question-answering technology, and large language models to derive context and answer queries. AiSay can scale through synchronous and asynchronous call options, allowing it to handle large file sizes. It is hosted serverlessly in the cloud, making it scalable in capacity to handle large file sizes. AiSay is actively being developed and supports RESTRICTED/SENSITIVE NORMAL levels of security and data classification. More document types will be supported in the future. AiSay’s unique features include its ability to provide structured responses for various document types, scalability, and support for security and data classification.“
Graph RAG: “AiSay is an AI-driven document reader developed by the Singapore government. It is a document parser that can provide structured responses, extracting and integrating information from various documents into existing systems. AiSay can automatically detect, scan, and understand various documents, including handwritten notes, printed articles, multilingual texts, and structured/unstructured documents. Unlike traditional optical character recognition (OCR) systems, AiSay can provide structured responses for documents such as images (JPEG, PNG, TIFF) and PDFs.“

Some key benefits of AiSay include:

1. Reducing manual errors and improving data accuracy by automatically filling in key data for form submissions from document uploads.

2. Enhancing the digital service experience, as AiSay can automatically fill in key data once the document is uploaded.

AiSay is unique in its ability to provide structured responses for various document types, supporting multiple formats and languages. It also provides a confidence score for each task to determine if additional support is needed, minimizing human intervention in tasks with high confidence levels. Additionally, due to its serverless hosting in the cloud, AiSay is scalable, capable of handling large file sizes and asynchronous extraction processes.

The advantages of Graph RAG over traditional RAG methods are evident in this case.

This response not only shows accuracy but is rich in context and relationships, providing a level of depth that standard RAG methods lack.

The secret of Graph RAG lies in its ability to analyze user queries, locate relevant data in the graph database, and incorporate this contextual insight into guiding LLM responses.

This approach leverages a connected information network (which traditional methods may overlook), resulting in a more nuanced understanding of queries.

Let’s consider a few more examples.

Q: “I need to remove sensitive information from some of my documents. What products does Govtech offer that have these capabilities?”

Basic RAG: “Govtech offers two products, Cloak and FileSG, that have functionalities to help remove sensitive information from documents.“
Graph RAG: “GovTech offers a product called Cloak, which is a compliant data de-identification central privacy toolkit. This product helps civil servants apply data de-identification techniques to datasets and review re-identification risks in accordance with guidelines.“

Q: “What is the difference between DIAB and SIaC?”

Basic RAG: “The provided documents do not directly compare DIAB (Data Infrastructure in a Box) with anything named SIaC, nor do they mention SIaC. Therefore, based on the given context, it is impossible to directly outline the differences between DIAB and SIaC. However, we can summarize the key aspects of DIAB as follows:

1. **Purpose and Development**: DIAB was developed by GovTech’s Data Science and Artificial Intelligence Department (DSAID) to serve as an accelerator for deploying and maintaining data infrastructure on the Government Commercial Cloud (GCC). It is specifically designed to assist government agencies in architecting, deploying, and securing data infrastructure.

2. **Focus on Data Infrastructure**: DIAB emphasizes building a consistent and scalable data infrastructure, allowing agencies to save time and reduce overhead when designing and implementing cloud infrastructure. This focus enables agencies to concentrate more on building the application layer.

3. **Use of Infrastructure as Code (IaC)**: DIAB utilizes IaC templates and reference architectures to assist in data engineering considerations, streamlining the process of setting up and managing data infrastructure.

Without information about SIaC, a comparison is impossible. The characteristics of DIAB highlight its role as a tool for government agencies to efficiently deploy and manage data infrastructure in the cloud, leveraging IaC for efficiency and scalability.

Graph RAG: “DIAB (Data Infrastructure) and SIaC (Secure Infrastructure as Code) are both products aimed at simplifying cloud infrastructure deployment for government agencies. However, they differ in specific focuses and functionalities.

DIAB serves as an accelerator for deploying and maintaining data infrastructure on the Government Commercial Cloud (GCC). It helps agencies design, deploy, and secure data infrastructure, focusing on data engineering considerations and providing Infrastructure as Code (IaC) templates. DIAB is designed to help agencies build consistent and scalable data infrastructure, with key advantages including ease of use, enhanced efficiency, built-in compliance, deployment flexibility, and future-proofing.

On the other hand, SIaC provides customizable templates for securely, compliantly, and automatically deploying cloud infrastructure. It focuses on the secure configuration of management services and tools from cloud service providers, offering modular and easily adaptable templates. SIaC’s key features include compliance with Singapore government standards, modular design for various use cases, integration with SGTS components, and cloud-native design.

In summary, while both DIAB and SIaC aim to simplify cloud infrastructure deployment for government agencies, DIAB specifically targets data infrastructure deployment and emphasizes data engineering considerations and IaC templates, while SIaC focuses on secure and compliant cloud infrastructure deployment, providing customizable templates and modular design.“

Conclusion

The transition from traditional retrieval-augmented generation (RAG) to Graph RAG represents a significant step in how we interact with large language models (LLMs).

This transition addresses a fundamental challenge: how to enhance LLMs’ ability to provide contextually accurate answers to complex queries.

When comparing the two approaches, the advantages of Graph RAG in handling contextually complex queries become evident, as traditional RAG techniques often fail to address contextually complex issues.

In contrast, Graph RAG leverages a more complex knowledge network, providing responses that capture the nuances of queries with deeper understanding.

However, Graph RAG is not a silver bullet that can solve all problems.

It still heavily relies on the quality, depth, and breadth of the underlying knowledge graph. In cases where the knowledge graph is limited or biased towards specific domains, the performance of Graph RAG may not surpass that of traditional RAG methods.

That said, this transition holds promise for creating AI systems that better reflect human thought and discovery.

Compiled from:

From Conventional RAG to Graph RAG | by Terence Lucas Yap | Government Digital Services, Singapore | Mar, 2024 | Medium

Recommended Similar Articles:

GRAPHrag

Knowledge-Enhanced RAG Full Lifecycle Based on Knowledge Graphs – Graph RAG

Graph RAG: Unlocking Knowledge Enhancement of Large Models RAG

From Traditional RAG to Graph RAG – When Large Models Meet Knowledge Graphs