A private ChatGPT for your office: Open WebUI + Ollama on your own hardware

Put Ollama and Open WebUI on one server with a capable GPU, and your team gets a ChatGPT-style assistant in the browser whose prompts and documents never leave your network. Ollama runs the models on the server's GPU. Open WebUI, started as a single Docker container, adds what an office needs: user accounts, groups and permissions, chat history, and question answering over your own documents. Staff open a web address on the internal network; nothing goes to an outside AI service as long as you use local models and keep Ollama's cloud models and Open WebUI's web search switched off. A server with one 24 GB graphics card runs today's popular 27–35B models for a small team, and you can have the whole setup running in an afternoon. The steps below follow the official Open WebUI (v0.12.0) and Ollama (v0.40.4) documentation as of October 2026.

What you are building

• Ollama runs on the server and loads models onto the GPU. It listens on port 11434 and, by default, only accepts connections from the server itself.
• Open WebUI runs in Docker on the same server, talks to Ollama, and serves the web interface on port 3000. Its data (users, chats, uploaded files) lives in a Docker volume called open-webui.
• Staff use a browser on the office network, or over a VPN when they are away. Nothing is exposed to the internet.

Step 0: Size the hardware

Two things decide the hardware: the model you want, and how many people will use it at the same time.

The model. Office work such as drafting, summarising and answering questions about documents benefits from the 27–35B class or larger, and document Q&A needs a longer context than casual chat. With a 16K context, a 24 GB card such as the GeForce RTX 4090 can run:

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

A 96 GB workstation card such as the RTX PRO 6000 Blackwell Workstation Edition moves you up to 70B dense models and 100B-plus mixture-of-experts models:

Estimates from our methodology, using data as of 2026-10-14. Real speed depends on the runtime, drivers and prompt length.

A Mac Studio with lots of unified memory is a quiet alternative for the large MoE models; see our Mac guide. For other combinations, use the calculator or our GPU buying guide.

Concurrency. Ollama processes one request per model at a time by default, and queues the rest. Raising OLLAMA_NUM_PARALLEL lets it work on several at once, but Ollama's documentation notes that memory scales with the number of parallel requests times the context length. A team of a few people who use the assistant on and off is usually fine with the defaults; if many people use it at the same moment and queues become a problem, a serving engine such as vLLM is built for that (see our runtime comparison).

Step 1: Install and configure Ollama

On a Linux server, install Ollama with the official script, which also creates a systemd service:

curl -fsSL https://ollama.com/install.sh | sh

Then set a few options for an office server. Open the service override:

sudo systemctl edit ollama.service

and add under [Service]:

[Service]
Environment="OLLAMA_NO_CLOUD=1"
Environment="OLLAMA_CONTEXT_LENGTH=16384"

OLLAMA_NO_CLOUD=1 disables Ollama's cloud models and web search, so every model runs on your server. Setting the context length explicitly matters in both directions: on GPUs with less than 24 GiB, Ollama's default is only 4K tokens, too short for documents; on cards with 48 GiB or more the default is 256K, which can reserve far more memory than you need. Apply the change and download a model that fits your GPU:

sudo systemctl daemon-reload
sudo systemctl restart ollama
ollama pull qwen3.6:27b

Ollama's API has no authentication, which is why it listens only on 127.0.0.1 by default. Keep it that way: only Open WebUI should talk to it.

Step 2: Run Open WebUI with Docker

Install Docker on the server, then generate a secret key. Open WebUI uses it to sign logins; if it changes when the container is recreated, everyone is logged out, so set it once and keep it somewhere safe:

openssl rand -hex 32

Start Open WebUI with the command from the official quick start, replacing your-secret-key with that value:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:main

What the options do: -p 3000:8080 serves the interface on port 3000; --add-host lets the container reach Ollama on the host; -v open-webui:... keeps all data in a named volume that survives updates; --restart always brings it back after a reboot. The first start takes about a minute. Follow the log until it reports Application startup complete:

docker logs -f open-webui

If Open WebUI cannot reach Ollama. On Linux, a container cannot reach a service bound only to the host's 127.0.0.1. Open WebUI's troubleshooting page offers two fixes. One is to run the container on the host network, so Ollama can stay private; the interface is then on port 8080:

docker run -d --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:main

The other is to make Ollama listen on all interfaces with OLLAMA_HOST=0.0.0.0:11434. If you do that, block port 11434 from the rest of the network with your firewall, because anyone who can reach it can use your models without logging in.

If you prefer Docker Compose, the quick start has an equivalent docker-compose.yml.

Step 3: Create the admin account and invite your team

Open http://<server-address>:3000 from a computer on the office network. The first screen asks you to create the admin account: the first account created becomes the administrator, so do this immediately, before anyone else can reach the server. (Alternatively, set WEBUI_ADMIN_EMAIL and WEBUI_ADMIN_PASSWORD before the first start, and the admin is created automatically.)

After that, sign-up is switched off. To let colleagues register:

1. Go to your avatar → Settings → Admin → Authentication and turn on New Sign Ups.
2. New accounts start as Pending; approve them under Admin Panel → Users.
3. Create groups (for example by department) under Admin Panel → Users → Groups, and set baseline rights for everyone under Default Permissions. Permissions add up: a user gets everything granted by any of their groups.
4. To restrict a model or knowledge base, set it to Private and grant read or write access to specific groups.

For larger organisations, Open WebUI supports single sign-on through OAuth/OIDC (including Google, Microsoft and GitHub) and LDAP.

Step 4: Let it answer questions about your documents

Open WebUI's knowledge bases turn it into a document assistant:

1. Go to Workspace → Knowledge → Create, and upload files.
2. In a chat, type # to pick a knowledge base, or attach it permanently to a model under Workspace → Models.
3. Retrieval settings such as chunk size live under Settings → Admin → Documents.

Documents are split and embedded on your server. By default Open WebUI uses a small local embedding model (sentence-transformers/all-MiniLM-L6-v2); it can also use an embedding model served by Ollama (RAG_EMBEDDING_ENGINE=ollama), which is worth testing if your documents are not in English. Make sure the chat model's context is long enough to read the retrieved passages; Open WebUI's documentation points to the per-model setting under Settings → Admin → Models → Advanced Params, and the Ollama context length you set in step 1 applies too.

Step 5: A checklist for keeping data in-house

• Local models only. Keep OLLAMA_NO_CLOUD=1, and do not add external API connections in Open WebUI unless you mean to.
• Web search off. It is off by default (ENABLE_WEB_SEARCH=False); turning it on sends search queries to an outside provider.
• Offline mode. Once the embedding model has been downloaded, OFFLINE_MODE=True stops update checks and model downloads from Hugging Face.
• Community sharing off. ENABLE_COMMUNITY_SHARING defaults to true and shows options to share to the Open WebUI community; set it to false for an internal tool.
• Decide on admin access to chats. ENABLE_ADMIN_CHAT_ACCESS defaults to true, so administrators can read users' chats. Set it (and ENABLE_ADMIN_EXPORT) to false if your policy requires, and tell staff either way.
• Analytics. The official Docker image already sets SCARF_NO_ANALYTICS=true, DO_NOT_TRACK=true and ANONYMIZED_TELEMETRY=false.
• Network. Open WebUI's hardening guide says it is designed for private, trusted networks. Do not publish it on the internet; use a VPN for remote staff, and put it behind a reverse proxy with HTTPS (the docs cover Nginx and Caddy).
• Encryption at rest. For production, the Open WebUI documentation recommends PostgreSQL with encrypted storage, or an encrypted file system if you stay with the default SQLite database.

You can pass these settings as -e NAME=value options in the docker run command; the environment variable reference lists them all.

Step 6: Back up and update

Everything Open WebUI stores is in the open-webui volume. Back it up with a throwaway container:

docker run --rm -v open-webui:/data -v $(pwd):/backup alpine tar czf /backup/openwebui-$(date +%Y%m%d).tar.gz /data

To update, remove the container (the volume and its data stay), pull the new image, and start it again with the same command as in step 2:

docker rm -f open-webui
docker pull ghcr.io/open-webui/open-webui:main

Database migrations only go forward, so always back up first. For a shared team instance, the documentation recommends pinning a specific version tag (ghcr.io/open-webui/open-webui:vX.Y.Z) and updating deliberately rather than automatically. Ollama updates by re-running its install script.

Licences: the software and the models

• Ollama is MIT-licensed.
• Open WebUI uses the Open WebUI License: BSD-3 plus a branding clause since version 0.6.6 (April 2025). You may deploy it for any number of internal users free of charge as long as the "Open WebUI" name and logo stay intact. Removing or changing the branding is allowed only for deployments with at most 50 users in any rolling 30-day period, with written permission, or under an enterprise licence.
• Models each have their own licence. Most popular open models allow commercial use, but check the terms of the one you deploy:

If you are in Taiwan

Interest in on-premises (地端) AI in Taiwan is driven largely by data protection, and a few official documents are useful context. The Executive Yuan's 2023 reference guideline on generative AI for government agencies bars staff from giving confidential or personal information to generative AI, but allows closed, on-premises deployments to be used according to classification level once the system environment's security has been confirmed. The AI Basic Act (人工智慧基本法), promulgated on 14 January 2026, lists privacy protection and data governance, including data minimisation, among its principles. The Personal Data Protection Act continues to apply to any personal data your staff put into the system. Taiwan's own TAIDE models, developed by the National Applied Research Laboratories, are published on Hugging Face under their own licence terms. Our Chinese edition of this guide covers these in more detail. None of this is legal advice: confirm your obligations with your legal or compliance team.