To self-host AI tools, you run open-source software such as Open WebUI, n8n, Ollama and LiteLLM on a server you control, and connect it either to local models or to hosted model APIs. This guide covers what each layer does, working Docker commands, the real costs, the maintenance you sign up for, and when self-hosting is not worth it.
- Self-hosting the interface and automation layer is cheap and easy; self-hosting the models themselves is where cost and effort jump.
- A practical stack: Open WebUI for chat, n8n for workflows, LiteLLM as a gateway with keys and budgets, and hosted APIs or Ollama for models.
- Small open models run on a CPU server; models that compete with frontier APIs need serious GPU memory.
- Put everything behind HTTPS and login, never expose admin ports to the internet, and back up the data volumes.
- Check licenses: n8n's Sustainable Use License allows internal business use but not reselling n8n as a service.
What does it mean to self-host AI tools?
Self-hosting AI tools means running the software yourself, on a server or computer you control, instead of using a vendor's hosted app. You can self-host any layer of an AI stack: the chat interface, the automation engine, a gateway that manages keys and spend, the models themselves, and coding agents.
The most useful thing to understand up front is that the layers have very different costs:
| Layer | Example tools | Hardware needed | Effort |
|---|---|---|---|
| Chat interface | Open WebUI, LibreChat | Small VPS | Low |
| Automation | n8n | Small VPS | Low to medium |
| Gateway | LiteLLM | Small VPS | Low |
| Retrieval over your docs | Built-in RAG in Open WebUI, pgvector | Small to medium VPS | Medium |
| Models | Ollama, llama.cpp, vLLM | CPU for small models, GPUs for large | Medium to high |
| Coding agents | Claude Code, Codex on your server | Medium VPS | Medium |
Most teams get the best value by self-hosting the top layers and calling hosted model APIs for the heavy lifting. Running your own models makes sense for strict privacy, offline use, or steady high volume.
Why self-host AI tools, and when shouldn't you?
Self-host when you need control: over data location, over costs per user, over which models people can use, or over workflows a SaaS tool cannot express. Do not self-host when nobody on the team wants to maintain servers.
Good reasons:
- One interface, many models. Give a team one chat UI connected to several providers and local models, with admin control over who uses what.
- Pay per use instead of per seat. A team with many occasional users can pay API rates instead of a subscription for everyone.
- Data control. Prompts and documents stay on your server, and with local models they never leave it.
- Custom automation. n8n workflows can mix AI steps with your internal systems in ways hosted tools may not allow.
Reasons to stay hosted:
- Maintenance. Updates, security patches, backups and outages become your job. n8n's own docs warn that self-hosting "requires technical knowledge" and that mistakes "can lead to data loss, security issues, and downtime."
- Features. Hosted assistants ship features (deep research, voice, mobile apps, agents) faster than self-hosted UIs replicate them.
- Model quality. Open models you can run on modest hardware are behind frontier hosted models on hard reasoning and coding tasks.
How do you self-host a ChatGPT-style interface?
Run Open WebUI in Docker and connect it to hosted APIs, local models, or both. It gives you a ChatGPT-like interface with user accounts, chat history and model selection, on your own server.
The command from the Open WebUI quick start, without a bundled model runner:
docker run -d -p 3000:8080 \ --add-host=host.docker.internal:host-gateway \ -v open-webui:/app/backend/data \ -e WEBUI_SECRET_KEY="$(openssl rand -hex 32)" \ --name open-webui --restart always \ ghcr.io/open-webui/open-webui:mainThen open http://localhost:3000 (or your server's address) and create the first account, which becomes the admin. A few things to know:
- Connections. Under Admin, Connections, add any OpenAI-compatible API by URL and key. The docs cover OpenAI, Anthropic and other providers.
- Local models. If Ollama runs on the same host, Open WebUI finds it at
http://host.docker.internal:11434. There is also an image tagged:ollamathat bundles Ollama in the same container. - Data. Everything (users, chats, uploaded files) lives in the
open-webuivolume. Back up that volume.
LibreChat is a popular alternative with a similar feature set. Pick one, since both solve the same problem.
How do you self-host AI automation with n8n?
Run n8n in Docker and use its AI nodes to call models from your workflows. n8n is a workflow automation tool, comparable to Zapier or Make, that you can host yourself.
The n8n Docker guide starts with a volume for persistent data and a container on port 5678:
docker volume create n8n_datadocker run -it --rm --name n8n -p 5678:5678 \ -e GENERIC_TIMEZONE="Europe/Berlin" -e TZ="Europe/Berlin" \ -v n8n_data:/home/node/.n8n \ docker.n8n.io/n8nio/n8nThat command is fine for trying n8n on your own machine. For a server, n8n recommends its Docker Compose setup with a reverse proxy for HTTPS, and points users without server experience to n8n Cloud.
n8n's self-hosted edition uses the Sustainable Use License. It permits use for your organization's internal business purposes, including building workflows for clients on your own instance. Hosting n8n and charging others for access, or white-labeling it, needs a commercial license.
For a full build of an AI agent workflow, see the n8n AI agent tutorial, and for choosing between tools, Zapier vs Make vs n8n.
How do you self-host the models themselves?
Use Ollama to download and run open models with one command, and size your hardware to the model. Ollama runs on Linux, macOS and Windows and exposes an API on port 11434.
curl -fsSL https://ollama.com/install.sh | shollama run gemma4The second command downloads the model on first use and opens a chat. Applications talk to Ollama through its native API or its OpenAI-compatible endpoint at http://localhost:11434/v1/, where the API key can be any placeholder. Most tools that accept an OpenAI base URL work with it unchanged.
Hardware is the real constraint. A rough rule for memory, derived from how quantization works:
| Model size (parameters) | Approx. memory at 4-bit quantization | Realistic hardware |
|---|---|---|
| 1 to 4 billion | 1 to 3 GB | Any modern laptop or small VPS (CPU) |
| 7 to 9 billion | 5 to 6 GB | 16 GB RAM CPU server (slow) or an 8 GB GPU |
| 12 to 14 billion | 8 to 10 GB | GPU with 12 to 16 GB |
| 27 to 32 billion | 17 to 20 GB | GPU with 24 GB |
| 70 billion and up | 40 GB and more | Multiple GPUs or high-memory workstations |
Each parameter at 4 bits takes half a byte, so a 7-billion-parameter model needs about 3.5 GB for weights plus room for context and runtime overhead. CPU inference works for small models but is slow for long answers; GPUs make larger models usable.
For serving many users at once, inference servers such as vLLM handle batching and throughput better than a desktop-oriented runner. The guide to running an LLM locally compares Ollama, LM Studio and llama.cpp in more depth.
How do you manage keys, costs and access with a gateway?
Put a gateway such as LiteLLM between your tools and the model providers. It gives every app one OpenAI-compatible endpoint, and lets you issue separate keys with their own budgets and rate limits.
According to the LiteLLM docs, the gateway supports more than 100 providers, records the cost of each request, and reports spend by key, team, user and tag, with load balancing and fallbacks between deployments. The quick start:
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.sh | shThe admin UI then runs at http://localhost:4000/ui. A typical setup:
- Add your provider keys (Anthropic, OpenAI, Google) and your Ollama endpoint to the gateway, once.
- Create a virtual key per app or team, each with a monthly budget.
- Point Open WebUI, n8n and your scripts at the gateway instead of at providers directly.
- Review spend per key weekly.
This keeps real provider keys in one place, so revoking access means deleting one virtual key, and a runaway workflow hits its budget instead of your credit card limit.
Can you self-host AI coding agents?
Yes. Coding agents such as Claude Code and Codex are command-line programs, so they run on any Linux server, where they keep working after you disconnect. The agent runs on your server, while the model behind it is still the vendor's hosted model, used through your subscription or API key.
Running agents on your own server gives you an always-on machine with your toolchain and full control over isolation. It also makes you responsible for SSH hardening, credentials, sandboxing and updates. The step-by-step setup is in the guide to a remote development server for AI agents, and the security side in prompt injection and coding agents.
If you want the always-on part without running the server yourself, Orkio rents an isolated cloud box where Claude Code and Codex keep working after you close your laptop, using your own Claude account.
What does self-hosting AI cost, and what upkeep does it need?
The software is free to run in most cases; you pay for servers, model usage and your time. Budget for all three before you start.
| Cost item | What drives it |
|---|---|
| VPS for UI, automation, gateway | Number of users and workflows; a small server runs all three for a small team |
| GPU servers for local models | Model size and hours running; the largest cost by far if you run big models |
| Hosted model APIs | Tokens used; check each provider's pricing page |
| Storage and backups | Chat history, uploaded files, vector indexes |
| Your time | Updates, incidents, user support |
A maintenance routine that keeps a self-hosted stack healthy:
- HTTPS and login on everything. Use a reverse proxy (Caddy, Traefik or nginx) with automatic certificates. Never expose Ollama's port 11434, the LiteLLM admin UI or n8n directly to the internet without authentication.
- Pin and update images. Pin Docker image versions, read release notes, and update monthly or when security fixes ship.
- Back up volumes. Open WebUI's data volume, n8n's
n8n_datavolume and the gateway's database hold everything that matters. Test a restore at least once. - Monitor disk. Model files are several gigabytes each; old models and Docker images fill disks quickly.
- Review access. Remove users and keys that are no longer needed.
An exposed, unauthenticated Ollama or n8n instance lets anyone on the internet run your models or your workflows, including workflows that hold credentials to other systems. Bind admin services to localhost or a private network.
How to start self-hosting AI tools this week
Start with the cheapest, most useful layer and add more only when you need it.
- Rent a small VPS or use a spare machine; install Docker.
- Run Open WebUI with the command above and connect one hosted API key.
- Add LiteLLM in front once a second app or a second team needs access, so keys and budgets live in one place.
- Add n8n when you have a recurring workflow worth automating.
- Try Ollama with a small model to see whether local models are good enough for any of your tasks.
- Put HTTPS and backups in place before inviting anyone else.
- Write down your setup in a script or Compose file so you can rebuild the server in under an hour.
Frequently asked questions
Is self-hosting AI cheaper than paying for ChatGPT or Claude?
For a team, self-hosting a chat interface connected to model APIs can cost less than per-seat subscriptions, because you pay per token used. Running your own models is rarely cheaper unless you have steady, high volume or already own GPUs. Include your time for maintenance when comparing.
What hardware do I need to self-host an LLM?
It depends on model size. As a rough rule, a model quantized to 4 bits needs about half a gigabyte of memory per billion parameters, plus overhead for context. Small models run on a CPU server with 8 to 16 GB of RAM, slowly; larger models need GPUs with enough memory to hold the whole model.
Is Open WebUI free?
Open WebUI is open-source software you can run yourself with Docker at no license cost for typical self-hosted use; check its license page for the exact terms. You still pay for the server and for any hosted model APIs you connect to it.
Can I self-host n8n for my company for free?
Yes, for internal use. n8n's self-hosted Community edition is under the Sustainable Use License, which allows use for your own organization's internal business purposes. Offering n8n itself to customers as a paid or white-labeled service needs a commercial license.
Is self-hosted AI more private?
Only if the models also run on your hardware. A self-hosted chat UI that calls a hosted API still sends prompts to that provider, under its data terms. Running models locally with Ollama keeps prompts on your server, at the cost of lower model quality for most tasks.
