Ollama + Open WebUI: Build Your Own Local ChatGPT Interface (Complete Guide)

Honestly, I was pretty nervous the first time I tried running an LLM locally. I kept thinking: Will this thing actually work? Will my old laptop just freeze up? Turns out—it went way smoother than I expected.

This article will show you how to set up a ChatGPT-style chat interface on your own machine using Ollama and Open WebUI. The whole process takes about 30 minutes.

---

Why Go Local?

ChatGPT and Claude are great, but a few things bother me.

The cost thing. Twenty bucks a month adds up to $240 a year. If you’re a heavy user, fine. But I just occasionally ask coding questions or look stuff up—that money feels wasted. Run models locally, and once downloaded, it’s completely free. No token billing, no subscription renewals to worry about.

Privacy concerns. Everything you type gets uploaded to OpenAI or Anthropic servers. Work documents, personal notes, private conversations—honestly, I’m not comfortable with that. Deploy locally, and all your data stays on your own hard drive. Nowhere else.

Offline access. Traveling, bad internet—cloud AI becomes useless. Local models run completely offline once downloaded. Works fine without any network.

Customization. Cloud services lock down the parameters. Temperature, Prompt templates—you can’t adjust any of it. Run locally, change whatever you want.

---

Core Concepts at a Glance

Let’s get clear on what these two tools are, or you’ll get confused later.

Ollama is a model runner. It helps you download, manage, and run large language models, and provides an API (default port 11434). Think of it as “a local version of OpenAI API”.

Open WebUI is a web interface. ChatGPT-style chat window, model switching, history management—it’s all there. It connects to Ollama via API, turning command-line operations into a graphical interface in your browser.

The architecture looks like this:

Browser (localhost:3000)
    ↓
Open WebUI (Docker container)
    ↓ HTTP API
Ollama (local service, port 11434)
    ↓
Local Models (stored in ~/.ollama)

Open browser → Open WebUI calls Ollama API → Ollama loads model and runs inference → Results come back to you. That simple.

---

System Requirements and Preparation

Before deploying, check if your hardware is up to the task.

Minimum requirements:

• Processor: Intel i5 or equivalent
• Memory: 8GB RAM (16GB+ recommended)
• Storage: At least 10GB free space (model files are large)
• OS: Windows 10+, macOS 11+, Linux

Recommended setup:

• GPU: NVIDIA RTX 3060 or better (inference is much faster)
• Apple Silicon Mac: M1/M2/M3 series, unified memory architecture, naturally suited for running models

Software dependencies:

• Docker: For running Open WebUI (you can skip Docker, but it’s easier with it)

Check if Docker is installed:

docker --version
docker compose version

If you see version numbers, it’s installed. If not, install Docker Desktop (Windows/macOS) or Docker Engine (Linux) first.

---

Step 1: Install Ollama

Ollama installation—one command and you’re done.

1.1 macOS and Linux Installation

Open terminal and run:

curl -fsSL https://ollama.com/install.sh | sh

The script automatically downloads Ollama and installs it to your system. After installation, the background service starts automatically.

1.2 Windows Installation

Run this in PowerShell:

irm https://ollama.com/install.ps1 | iex

Or just go to ollama.com/download, download the Windows installer, and double-click to run. Either way works.

1.3 Docker Installation (Optional)

Want to put Ollama in a container too? Use Docker:

docker pull ollama/ollama:latest
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

-v ollama:/root/.ollama stores models in a Docker volume—models won’t be lost when the container restarts.

1.4 Verify Installation

After installation, confirm Ollama is running properly:

ollama --version

You should see output like ollama version is 0.1.x.

If the service didn’t start automatically, run it manually:

ollama serve

This starts the Ollama API service in the background, listening on http://localhost:11434.

---

Step 2: Download and Run Models

Ollama is installed. Now let’s download a model.

2.1 How to Choose a Model

Different model sizes require different hardware. Use this table to pick quickly:

Hardware	Recommended Model	Parameters	Disk Space	Command
8GB RAM, no GPU	Llama 3.2 1B	1B	740MB	ollama run llama3.2:1b
8GB RAM, no GPU	Gemma 3 2B	2B	1.4GB	ollama run gemma3:2b
16GB RAM	Llama 3.2 3B	3B	2GB	ollama run llama3.2
16GB RAM	Qwen 2.5 7B	7B	4GB	ollama run qwen2.5:7b
16GB RAM + GPU	Llama 3.1 8B	8B	4.7GB	ollama run llama3.1:8b
32GB RAM + GPU	DeepSeek R1 14B	14B	8GB	ollama run deepseek-r1:14b
64GB RAM + GPU	Llama 3.3 70B	70B	39GB	ollama run llama3.3:70b

Choose by use case:

• Daily chat, simple tasks: Llama 3.2 1B or 3B
• Better Chinese: Qwen 2.5 series (from Alibaba, great Chinese performance)
• Coding: DeepSeek R1 series (strong reasoning, good for code generation)
• General high performance: Llama 3.1 8B or Mistral 7B

2.2 Download Models

Use the ollama pull command:

# Download Llama 3.2 (default 3B version)
ollama pull llama3.2

# Download DeepSeek R1 7B
ollama pull deepseek-r1:7b

# Download Qwen 2.5 7B (good for Chinese)
ollama pull qwen2.5:7b

First download might take a few minutes—depends on model size and your internet speed. Llama 3.2 took me about 2 minutes, DeepSeek R1 7B took a bit longer.

2.3 Run Model Chat

Once downloaded, run it directly:

ollama run llama3.2

You’ll enter a chat interface:

>>> Send a message (/? for help)

Type your question, the model responds in real-time:

>>> Hello, introduce yourself

Hello! I'm a local AI assistant based on the Llama 3.2 model...

Exit with Ctrl + d or type /bye.

2.4 Common Management Commands

See downloaded models:

ollama list

Delete a model:

ollama rm llama3.2:1b

Copy a model (create an alias):

ollama cp llama3.2 my-llama

Show model details:

ollama show llama3.2

---

Step 3: Install Open WebUI

Command-line works fine. But if you want a ChatGPT-style graphical interface—then install Open WebUI.

3.1 Docker Single Container Deployment (Super Fast)

First make sure Ollama service is running (ollama serve), then execute:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart unless-stopped \
  ghcr.io/open-webui/open-webui:main

Parameter breakdown:

• -p 3000:8080: Maps container’s 8080 port to your machine’s 3000 port
• --add-host=host.docker.internal:host-gateway: Lets container access host’s Ollama service
• -v open-webui:/app/backend/data: Persists data (chat history, user accounts)
• --restart unless-stopped: Container auto-starts after Docker restart

Once deployed, open browser and visit:

http://localhost:3000

3.2 Docker Compose Deployment (Recommended)

Want to manage both Ollama and Open WebUI with Docker? Docker Compose is more convenient.

First create a directory:

mkdir open-webui-project
cd open-webui-project

Create a compose.yaml file:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - ./data:/app/backend/data
    environment:
      - "OLLAMA_BASE_URL=http://ollama:11434"
    restart: unless-stopped
    depends_on:
      - ollama

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ./ollama:/root/.ollama
    ports:
      - "11434:11434"
    restart: unless-stopped

Start services:

docker compose up -d

Check status:

docker compose ps

Both containers showing running means success.

3.3 First Access and Configuration

Open browser and visit http://localhost:3000. First time you’ll see the account creation page.

Create admin account:

• Enter username, email, password
• Click “Sign Up”

After logging in, Open WebUI automatically detects Ollama service and lists available models. If you’ve already downloaded models, the dropdown will show something like:

llama3.2:latest
deepseek-r1:7b
qwen2.5:7b

Not auto-detected? Go to Settings → Connections, manually enter Ollama API address:

http://host.docker.internal:11434

If using Docker Compose, enter:

http://ollama:11434

---

Step 4: Basic Usage

Interface is set up. Let’s see how to use it.

4.1 ChatGPT-Style Chat

Interface looks a lot like ChatGPT:

• Left side is chat list, can create, delete, rename
• Top is model selection dropdown
• Bottom is input box, press Enter after typing

Select a model (like llama3.2), type your question, response streams in real-time—just like ChatGPT. Pretty smooth.

4.2 Model Switching

You can switch models mid-conversation. Click the model dropdown at top, select another model, continue chatting.

This feature is great for comparing different models’ responses. For example:

• Ask same question, let Llama 3.2 answer first
• Switch to DeepSeek R1, see if its answer goes deeper

4.3 Chat Management

• New chat: Click “New Chat” on the left
• Rename chat: Click chat title, edit directly
• Delete chat: Right-click chat, select delete
• Search history: Search box at top left

All chats are stored in local Docker volume, not uploaded to cloud.

4.4 Parameter Tuning

Click the settings icon to the right of input box to adjust model parameters:

• Temperature: Controls output randomness. Low values (0.1-0.3) more stable, high values (0.7-0.9) more creative
• Top P: Controls vocabulary selection range
• Max output length: Limit response length

These parameters significantly affect results. Use low Temperature for coding, high for creative writing.

---

Step 5: Advanced Features

Beyond basic chat, Open WebUI has several practical advanced features.

5.1 RAG Knowledge Base Building

RAG (Retrieval-Augmented Generation)—turn your documents into an AI-searchable knowledge base.

How to use:

1. Click “Documents” tab on the left
2. Click “Upload”, select files (supports PDF, Markdown, TXT, DOCX)
3. After upload, system automatically processes and builds index

Once uploaded, check “Use Documents” during chat, AI will retrieve relevant info from your documents.

Example scenarios:

• Upload company API docs, ask “what are the parameters for endpoint X”
• Upload personal notes, ask “what was the conclusion from last meeting”
• Upload technical PDF, ask “detailed explanation of concept X”

This is local “knowledge base chat”—all documents stay on your own hard drive.

5.2 Multi-User Management

Multiple people using the system? You can create different accounts.

Admins can in Settings → Users:

• Create new users
• Set user permissions (regular user, admin)
• View user list

Each user’s chat history is stored separately, no interference.

5.3 API Integration

Both Open WebUI and Ollama provide OpenAI-compatible APIs that can directly connect to existing tools.

Call Ollama API:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {"role": "user", "content": "Hello"}
  ],
  "stream": false
}'

Call Open WebUI API (if authentication needed, get Token first):

curl http://localhost:3000/api/chat/completions \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{
    "model": "llama3.2",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Python example:

import requests

response = requests.post('http://localhost:11434/api/chat', json={
    'model': 'llama3.2',
    'messages': [{'role': 'user', 'content': 'Hello'}],
    'stream': False
})

print(response.json()['message']['content'])

This way, you can connect local models to VS Code plugins, automation scripts, or any tools that support OpenAI API.

---

Step 6: Performance Tuning

Inference too slow? Not enough memory? Try adjusting a few parameters.

6.1 GPU Acceleration

Ollama automatically detects and uses GPU. GPU not recognized? Check if drivers are installed correctly.

NVIDIA GPU:

Make sure NVIDIA drivers and CUDA are installed. Ollama automatically calls nvidia-smi to detect GPU.

Apple Silicon Mac:

M1/M2/M3 Mac needs no extra configuration—Ollama automatically uses Metal acceleration.

AMD GPU:

On Linux you can install the ROCm version of Ollama:

curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz -o ollama-linux-amd64-rocm.tgz
sudo tar -C /usr/ -xzf ollama-linux-amd64-rocm.tgz

6.2 Concurrency Settings

Want multiple models running simultaneously, or improve concurrent processing? Set environment variables:

# Set number of parallel models
OLLAMA_NUM_PARALLEL=2 ollama serve

# Set max loaded models
OLLAMA_MAX_LOADED_MODELS=2 ollama serve

These parameters work well for multi-user scenarios, avoiding load delays when frequently switching models.

6.3 Model Quantization

Large models (like 70B) using too much memory? Use quantized versions to reduce usage:

# Download 4-bit quantized version (usage reduced by ~75%)
ollama pull llama3.3:70b-q4_K_M

Quantization slightly reduces quality, but speed and memory usage improve significantly.

---

Common Issues and Solutions

You might encounter some issues during deployment. Here are the most common ones.

Q1: Model Download Very Slow

Cause: Ollama’s official servers are overseas, downloads might be slow in some regions.

Solution:

• Use a proxy
• Or download GGUF files from mirror sites, manually import to Ollama

Manual import method:

# After downloading GGUF file, create a Modelfile
FROM ./llama3.2.gguf

# Create model
ollama create my-llama3.2 -f Modelfile

Q2: Open WebUI Can’t Connect to Ollama

Symptom: Interface shows “Ollama connection failed”.

Troubleshooting:

1. Confirm Ollama service is running:

curl http://localhost:11434

Should return "Ollama is running".

2. If using Docker, check network configuration:

docker exec -it open-webui curl http://host.docker.internal:11434

If this doesn’t work, you might need to manually configure OLLAMA_BASE_URL environment variable.

Q3: GPU Not Recognized

Symptom: Inference very slow, nvidia-smi shows GPU not being used.

Troubleshooting:

1. Check NVIDIA drivers:

nvidia-smi

Should display GPU information.

2. Check if Ollama recognizes GPU:

ollama show llama3.2 --system

If it shows CPU-only, GPU isn’t recognized.

Solution:

• Reinstall NVIDIA drivers
• Confirm CUDA version compatibility
• On Linux might need to set CUDA_VISIBLE_DEVICES

Q4: Out of Memory Error

Symptom: Running large model throws OOM (Out of Memory) error.

Solution:

• Use smaller model (like 1B or 3B)
• Use quantized version (q4_K_M)
• Increase swap space (Linux)
• Close other memory-consuming programs

Q5: Docker Container Startup Failure

Symptom: docker ps shows container status as Exited.

Troubleshooting:

View container logs:

docker logs open-webui

Common causes:

• Port conflict (port 3000 used by other program)
• Volume permission issues
• Insufficient memory

Solution:

• Use different port (like -p 8080:8080)
• Check Docker volume permissions
• Increase container memory limit

---

Summary

Setting up a ChatGPT interface locally with Ollama + Open WebUI involves just these core steps:

1. Install Ollama (one command)
2. Download appropriate model (choose based on hardware)
3. Deploy Open WebUI (Docker one-click startup)
4. Start chatting

Once set up, you have:

• Free AI chat assistant
• Completely private data storage
• Works offline
• Customizable model parameters
• Extensible knowledge base (RAG)
• Integrable API interface

This setup works great for personal daily use and small team internal deployment. If you want to go deeper, explore:

• Modelfile for customizing model behavior
• Multi-cloud model integration (use local models alongside OpenAI API)
• Kubernetes production-grade deployment

---

Build Local ChatGPT Interface

Deploy a ChatGPT-style AI chat interface locally with Ollama and Open WebUI

⏱️ Estimated time: 30 min

1. 1

   Step1: Install Ollama

   Choose installation method based on your OS:
   
   • macOS/Linux: curl -fsSL https://ollama.com/install.sh | sh
   • Windows: irm https://ollama.com/install.ps1 | iex
   • Docker: docker pull ollama/ollama:latest
   
   After installation, run ollama --version to verify
2. 2

   Step2: Download Model

   Choose model based on your hardware:
   
   • 8GB RAM: ollama pull llama3.2:1b
   • 16GB RAM: ollama pull llama3.2
   • 16GB RAM + GPU: ollama pull llama3.1:8b
   • Chinese optimized: ollama pull qwen2.5:7b
3. 3

   Step3: Deploy Open WebUI

   Docker single container deployment:
   
   docker run -d -p 3000:8080 \
   --add-host=host.docker.internal:host-gateway \
   -v open-webui:/app/backend/data \
   --name open-webui \
   ghcr.io/open-webui/open-webui:main
   
   Or use Docker Compose to manage dual containers
4. 4

   Step4: Initial Configuration

   Visit http://localhost:3000:
   
   • Create admin account
   • Auto-detect Ollama service
   • Select model and start chatting
5. 5

   Step5: Advanced Feature Setup

   Optional features:
   
   • RAG knowledge base: upload PDF/Markdown documents
   • API integration: OpenAI-compatible endpoint
   • GPU acceleration: auto-detected, no config needed

FAQ

What hardware do I need for local AI deployment?

Minimum 8GB RAM, Intel i5 processor, 10GB storage. Recommended 16GB+ RAM, NVIDIA RTX 3060 or Apple Silicon Mac for better performance. Large models (like 70B) require 64GB RAM.

Which models does Ollama support?

Supports mainstream open-source models:

• Meta Llama series (1B-70B)
• DeepSeek R1 reasoning models
• Alibaba Qwen series (Chinese optimized)
• Google Gemma, Mistral, and more

Use ollama list to see downloaded models, ollama pull to download new ones

What if Open WebUI can't connect to Ollama?

First confirm Ollama service is running (curl http://localhost:11434). For Docker deployment, check network configuration, may need to manually set OLLAMA_BASE_URL environment variable to http://host.docker.internal:11434

Can locally deployed AI work offline?

Yes. Once models are downloaded, they run completely offline without network connection. Chat history and knowledge base documents are all stored locally, perfect for travel or unstable network environments

How do I integrate with existing tools like VS Code plugins?

Ollama provides OpenAI-compatible API (port 11434):

• Endpoint: http://localhost:11434/api/chat
• Same format as OpenAI API
• Direct calls from Python/Node.js

Just change the API address from OpenAI to your local address

---

Resources

• Ollama Official Site
• Ollama Model Library
• Open WebUI GitHub
• Ollama GitHub
• freeCodeCamp Tutorial
• DataCamp Open WebUI Tutorial

Got questions? Search the Ollama and Open WebUI GitHub Issues—active community, most problems have been answered before.

11 min read · Published on: Apr 4, 2026 · Modified on: Apr 5, 2026