Project Introduction
-
Project: There are various types of agents (projects), each with its own functionality. (rag, ragsql, inference, vision)
-
User: Users represent the system’s users. It is used for authentication and authorization (basic authentication). Each user can access multiple projects.
-
LLMs: Supports any public or local LLM supported by LlamaIndex or Ollama.
-
VRAM: Automatic VRAM management. RestAI will manage VRAM usage, automatically loading and unloading models as needed and requested.
-
API: The API is a first-class citizen of RestAI. All endpoints are documented using Swagger.
-
Frontend: restai-frontend has a usable frontend.
Project Types
RAG
-
Embedding: You can use any embedding model supported by llamaindex. Check the embedding definition.
-
Vectorstore: Supports two vector stores:
<span>Chroma</span>and<span>Redis</span> -
Retrieval: It has embedding search and scoring evaluators that allow you to assess the quality of embeddings and in LLM. It also supports re-ranking, ColBERT, and LLM-based.
-
Loader: You can use any loader supported by llamaindex.
-
Sandbox Mode: The RAG agent (project) has a “sandbox” mode, which means that when a provided question has no embedding, a locked default answer will be given. This is useful for chatbots, where in an LLM chatbot, when it doesn’t know how to answer a question, you want to provide a default answer to avoid hallucination.
-
Evaluation: You can use deepeval to evaluate your RAG agent. Use the
<span>eval</span>attribute in the RAG endpoint.
RAGSQL
-
Connection: Provide a MySQL or PostgreSQL connection string, which will automatically crawl the database schema, using table and column names to figure out how to convert questions into SQL and then write responses.
Inference
Vision
-
text2img: RestAI supports local Stable Diffusion and Dall-E. It has a prompt enhancement feature that uses LLM internally to enhance user prompts with more details.
-
img2text: RestAI supports LLaVA and BakLLaVA by default.
-
img2img: RestAI supports InstantID and Qwen-VL by default.
Stable Diffusion & InstantID
LLaVA Company
LLMs
-
You can use any LLM product supported by Ollama and/or LlamaIndex.
Installation
-
RestAI uses Poetry to manage dependencies. Use
<span>pip install poetry</span>.
Project Links
https://github.com/apocas/restai
Follow the “GitHubStore” public account
Scan the WeChat below
1 Join the technical exchange group, note “Programming Language-City-Nickname“
