Deploy Your Local Large Language Model Knowledge Base with Ollama + Open WebUI

Install and Set Up Ollama

https://ollama.com/

After downloading and installing, it will automatically start with the system by default. The model is installed on the default system disk, so there is no need to set it to auto-start; you can start it whenever you want; then set the default installation path for the model.

Delete the Ollama shortcut in the directory <span>C:\Users\</span>xxx\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup

Open the system environment variables, create a new variable <span>OLLAMA_MODELS</span> with the value: your custom model installation path.

Then restart <span>Ollama</span>.

Install Open WebUI

• Install Miniconda:

https://docs.anaconda.com/miniconda/install/

• Install Open WebUI:

https://github.com/open-webui/open-webui

Create a virtual environment named <span>openwebui</span> in the folder where you want to place <span>Open WebUI</span>:

<span>python -m venv openwebui</span>

Activate the environment:

<span>openwebui\Scripts\activate</span>

Install using Tsinghua mirror:

<span>pip install -i https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple open-webui</span>

Start <span>Open WebUI</span>:

<span>open-webui serve</span>

Once started, open the browser at: <span>http://localhost:8080</span>

Create an admin account (this is local, it won’t upload to the internet):

If you don’t need the OpenAI API, turn it off, or else you will get an error when starting (but it won’t affect functionality):

At this point, you can go to <span>https://ollama.com/search</span> to download various open-source large models to play with locally. Upload your private files to build your own knowledge base.

What if you want to play with models not in the Ollama model library (uncensored, NSFW)?

Create Your Own Ollama Model

Official tutorial:

https://github.com/ollama/ollama/blob/main/docs/import.md

Here is an example using the gguf model:

• <span>https://hf-mirror.com/models</span> Download your favorite model and place it in the Ollama model installation directory

• In the same directory, create a new file <span>Modelfile.txt</span> and input <span>FROM ./model_name.gguf</span>

• In the same directory, open cmd and execute <span>ollama create my_model -f Modelfile.txt</span>

• The newly created <span>my_model</span> model can now be used in <span>Open WebUI</span>:

Quick Start

Create a <span>run.bat</span> file, input the following content, place it in the previously created environment <span>openwebui</span> directory, and double-click it to start <span>Open WebUI</span>:

chcp 65001

@echo off
echo "      ___           ___           ___                ";
echo "     /\  \         /\  \         /\__\        ___    ";
echo "    |::\  \       _\:\  \       /:/ _/_      /\__\   ";
echo "    |:|:\  \     /\ \:\  \     /:/ /\  \    /:/__/   ";
echo "  __|:|\:\  \   _\:\ \:\  \   /:/ /::\  \  /::\  \   ";
echo " /::::|_\:\__\ /\ \:\ \:\__\ /:/_/:/\:\__\ \/\:\  \  ";
echo " \:\~~\  \/__/ \:\ \/:/  / \:\/:/ /:/  /  ~~\:\  \ ";
echo "  \:\  \        \:\ \::/  /   \::/ /:/  /      \:\__\";
echo "   \:\  \        \:\/:/  /     \/_/:/  /       /:/  /";
echo "    \:\__\        \::/  /        /:/  /       /:/  / ";
echo "     \/__/         \/__/         \/__/        \/__/  ";

REM 1. Activate environment
call "%~dp0Scripts\activate"

REM 2. Start
open-webui serve

REM 3. Optional: Keep window open
pause