Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities

Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities

by

Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities

Author: Liu Ran

Affiliation: China People’s Police University, Information Technology and Network Management Department

Editorial Note

In the wave of digital transformation, the information construction of universities is in full swing, with AI applications represented by large language models becoming an important topic for the construction of smart campuses. The One-Stop Service Platform, as an important window for universities to provide comprehensive services, directly affects the service experience of teachers and students. This article will introduce how to deploy intelligent customer service AI applications for the One-Stop Service Platform in universities using open source frameworks FastGPT, OneAPI, and Ollama.
Introduction

With the continuous advancement of artificial intelligence technology, universities have an increasing demand for improving service efficiency and quality. Intelligent customer service systems, as an important means to enhance service efficiency, have broad application prospects in the One-Stop Service Platform. By deploying intelligent question-and-answer AI applications, it can not only alleviate the pressure on human customer service but also provide uninterrupted service for teachers and students 24 hours a day.

1. Introduction to RAG Intelligent Q&A System
The RAG technology combines two stages: Retrieval and Generation, making it particularly suitable for tasks that require a large amount of background knowledge or specific dataset information. In the Q&A system, the RAG model first retrieves relevant information from a knowledge base related to the question, and then generates an answer based on that information.
2. System Architecture
1. FastGPT
FastGPT is a knowledge base Q&A system based on LLM (Large Language Model). It provides functions such as data processing, model invocation, RAG (Retrieval-Augmented Generation) retrieval, and visual AI workflow orchestration. It is suitable for building AI customer service in specific fields, such as intelligent Q&A systems for universities, business AI assistants, and teaching and research AI assistants.
2. OneAPI
OneAPI is an open-source AI interface management and distribution system that integrates multiple model interfaces to provide a unified invocation method for upper-level applications. OneAPI allows users to easily manage and invoke multiple different AI models, including but not limited to OpenAI, Tongyi Qianwen, Wenxin Yiyan’s official models, local models provided by Ollama, and customized models integrated with online knowledge bases.
3. Ollama
Ollama is a local large language model running framework that allows users to run large language models locally, such as Llama 3, Tongyi Qianwen, and other generative models and vector models.

System Architecture Diagram:

Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities
3. Deployment Process

1. Preparation

  • Environment Setup: Ensure that the server environment meets the operating requirements of FastGPT, OneAPI, and Ollama. For the test environment, we used 2 servers with 32G video memory and 1 private cloud host with 16C64G configuration.

  • Model Selection: Choose appropriate large language models, vector models, and result re-ranking models. In this experiment, we used Alibaba Cloud’s open-source Tongyi Qianwen 1.5’s qwen:32b as the text generation model, Nomic AI’s open-source nomic-embed-text as the vector model, and BAAI’s open-source BGE Re-Ranker v2 as the re-ranking model, which has been tested and is effective.

2. Deploying Ollama

  • Install Ollama: Download and install Ollama according to the official documentation.

  • Download and Run the Model: Use Ollama to run the required language model. The re-ranking model in this article is run separately using the open-source project script.

  • The recently updated version of Ollama supports parallel running of multiple models; interested users can try it.

3. Deploying OneAPI

  • Container Deployment: Use Docker to deploy OneAPI, configure port mapping and data volumes.

  • Interface Management: Configure the local model channels and tokens in OneAPI to ensure correspondence with the models in Ollama.

4. Deploying FastGPT
  • Deploy FastGPT using Docker Compose, modifying the docker-compose.yml and config.json files to connect the three models.
Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities
  • Prepare Data: Collect common inquiry questions and business process assistance for the One-Stop Service Platform.
  • Create Knowledge Base: Create a knowledge base in FastGPT and import the prepared data.
  • Create Application: Create a new application in FastGPT and introduce the knowledge base.
4. System Testing and Optimization
  • Conduct comprehensive testing of the deployed system, including functional testing, performance testing, and user experience testing.
  • Optimize knowledge base data based on testing feedback to improve system performance.
5. Measured Results

Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities

6. Conclusion and Outlook
The above system is fully localized and can run on a local area network. By combining FastGPT, OneAPI, and Ollama, universities can quickly deploy a stable, efficient, and easy-to-manage intelligent Q&A system. The implementation of the RAG architecture large model can not only be used as intelligent customer service but also as a business AI assistant and teaching research assistant. At the same time, we are also working on AI data applications based on natural language to SQL, looking forward to exploring more AI application scenarios in universities together.
References:
1. https://github.com/labring/FastGPT
2. https://github.com/songquanpeng/one-api
3. https://github.com/ollama/ollama
University Informationization Experts Exchange: “Call for Papers on University Informationization”

Previous Recommendations

  • Northwest Normal University: The school held a meeting to solicit opinions on the educational digitization action plan (2024-2028)

  • Liu Zheng: Serving AI application scenarios with open source – Beixin Science and Technology open-source large model application BistuCopilot

  • Beijing University of Posts and Telecommunications: V2.0 is back! Large model empowers teaching experiments to start comprehensively!

  • “i Huadian” intelligent assistant is online, inviting you to experience it~

  • Peking University: Recent guidelines for cleaning servers infected with Trojans

  • Minzu University of China: National Security Education Day | “Phishing Email” drill, did you fall for it?

  • Minzu University of China: Notice on strengthening password management of smart portals

  • Ningxia Medical University Network Information Center held a meeting on campus network security and data security work

  • [Security Knowledge] Changes in the requirements of the 2.0 standard under the equal protection

About Us

Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities

Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities

University Informationization Experts Exchange

University Informationization Experts Exchange (EDUITS) is a one-stop service platform for information technology in higher education, founded by a team of industry experts from Beijing Zhangyun Technology Co., Ltd. It aims to disseminate information on “smart campuses, cloud computing, network security, trusted products, industry policies, conference exchanges, expert papers, hot technologies, application scenarios, products, and solutions”. Additionally, the University Informationization Experts Exchange WeChat public account provides free recruitment information release services for network centers across the country. Please contact us: WeChat ID zyunits

Using Open Source Frameworks to Deploy Private RAG AI Applications in Universities

Technical Support: Beijing Zhangyun Technology Co., Ltd.

About Us Disclaimer

Leave a Comment