TrendPulse

Daily deep-dives on what's trending in tech

2026-09-24

How to Run Ollama on a Synology NAS (2026): Self-Hosted AI with Docker + Open WebUI

Your Synology NAS is already on 24/7, already sits on your LAN, and already holds your data. That makes it a surprisingly good host for a small, private AI endpoint — no API keys, no per-token bills, and nothing leaving your network. This guide walks through the exact setup I'd use on a Plus-series Synology: Ollama in Container Manager, Open WebUI as a ChatGPT-style front end, and honest numbers on what NAS-class hardware can actually do.

Let's set expectations up front: a NAS will not replace a GPU workstation. CPU-only inference on a Celeron or Ryzen embedded chip delivers roughly 2–8 tokens/second on 3B-parameter models. That's genuinely useful for document summarization, Home Assistant integration, and background automation — and too slow for snappy interactive chat with big models. If you want 70B-class local AI, you want a GPU box (see our RTX 5090 vs RTX 4090 comparison). If you want an always-on private assistant that costs ~$0 extra, keep reading.

Which Synology Models Can Run Ollama?

The hard requirement is an x86-64 CPU. Ollama publishes amd64 and arm64 Docker images, but the standard image assumes SIMD extensions that Synology's Realtek ARM chips don't have. That rules out the DS124, DS223, DS223j, and every other Realtek/Marvell-based value model.

What works: the Plus series and up, which ship Intel Celeron or AMD Ryzen embedded CPUs.

Model CPU Stock RAM Max RAM Verdict for LLMs
DS423+ / TS-464 class Intel Celeron N5095 2–4 GB 16 GB 3B models only, 2–5 tok/s
DS925+ AMD Ryzen R1600 4 GB 32 GB 3B comfortably, 7B at the edge
DS1522+ / DS1525+ AMD Ryzen 8 GB 32 GB 3B–7B models, 5–8 tok/s
DS1821+ AMD Ryzen V1500B 4 GB 32 GB 3B–7B, best headroom
DS124 / DS223 / DS223j Realtek ARM 1–2 GB — ❌ Not compatible

RAM is the second gate. Budget from free RAM after DSM and your other packages, not the sticker number. Practical rule: 6–8 GB free for a 3B model, 12–16 GB free for a 7B. If your NAS has 4 GB stock, a RAM upgrade is the single best money you can spend on this project — far better value than a bigger CPU.

The AVX2 factor

llama.cpp (the engine under Ollama) leans hard on wide SIMD instructions. A CPU with AVX2 runs noticeably faster than one without. Check yours over SSH:

lscpu | grep -E 'Model name|avx2'
free -g   # how much RAM is actually available

Older Celeron J-series chips without AVX2 will technically run models — unpleasantly.

Step 1: Install Container Manager

DSM 7.2 replaced the old Docker package with Container Manager. Install it from Package Center, then in File Station create two folders:

If you have an SSD volume or SSD cache, put the ollama folder there. Model weights are read in full on every cold start, and a 2 GB read off spinning disks is the difference between a 5-second and a 30-second wake-up.

Step 2: Deploy Ollama with Docker Compose

Container Manager accepts a Compose project directly, which is easier to reproduce than clicking through the GUI. Create docker-compose.yml:

services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - /volume1/docker/ollama:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0
      - OLLAMA_KEEP_ALIVE=-1
      - OLLAMA_MAX_LOADED_MODELS=1
    deploy:
      resources:
        limits:
          memory: 6g

Three settings matter more on a NAS than anywhere else:

In Container Manager → Project → Create, point at the folder holding the Compose file and start it.

Step 3: Pull a Model and Test

SSH in (sudo -i for root) and pull a small model first:

docker exec -it ollama ollama pull llama3.2:3b

That's a ~2 GB download. Test from any device on your LAN:

curl http://NAS-IP:11434/api/generate -d '{
  "model": "llama3.2:3b",
  "prompt": "Reply with one word: ready",
  "stream": false
}'

Which models are realistic on NAS hardware?

Model Size (Q4) RAM needed Good for
llama3.2:3b ~2 GB ~6 GB free General chat, summarization
phi3:mini (3.8B) ~2.3 GB ~6–8 GB free Efficient all-rounder
qwen2.5:3b / qwen3:4b ~2 GB ~6 GB free Multilingual, coding
mistral:7b ~4.4 GB ~10 GB free Better prose — DS1522+ class only
Anything 13B+ 8 GB+ — ❌ Don't. It will swap and crash.

The bandwidth math explains why: dual-channel DDR4-2666 peaks around 42.7 GB/s, and generation speed is bounded by bandwidth ÷ model size. A 3B Q4 model (~1.8 GB of weights) tops out around 23 tok/s theoretical; an 8B (~4.9 GB) around 8.7 tok/s — and real-world numbers land meaningfully below those ceilings. Single-channel RAM halves them again, so if you upgrade RAM, install matched pairs.

Step 4: Add Open WebUI (the ChatGPT-Style Interface)

The raw API is great for automation, but for daily use you want a chat UI. Add a second service to the same Compose file:

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    depends_on:
      - ollama
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://NAS-IP:11434
    volumes:
      - /volume1/docker/open-webui:/app/backend/data
    restart: unless-stopped

Use your NAS's actual LAN IP in OLLAMA_BASE_URL, not localhost — inside the Open WebUI container, localhost is the container itself. This is the #2 "it doesn't work" cause.

Browse to http://NAS-IP:3000, create an admin account (stored locally, no cloud), and every device in your house now has a private ChatGPT-style assistant.

Step 5: Security — Do Not Skip This

Ollama's API has no authentication. Anyone who can reach port 11434 can run queries and eat your NAS's resources.

Never expose bare Ollama to the public internet.

Troubleshooting the Common Failures

Is a NAS the Right Tool, or Should You Buy a GPU?

The honest framing: NAS inference wins on amortization, not speed. Your NAS already runs 24/7 at ~30–35 W under inference load — roughly $40/year in electricity. A 300 W GPU rig doing 8 hours/day runs closer to $390/year, plus $1,500–2,500 upfront.

Choose the NAS path for overnight document indexing, Home Assistant reasoning, private batch jobs, and casual Q&A. Choose a GPU (even a used RTX 3060 12GB at ~$270–300 delivers 16–20 tok/s on 8B coding models) when you want interactive speed. QNAP owners with PCIe slots (TS-473A, TS-673A) can even put a low-profile GPU inside the NAS and jump to 40–80 tok/s on 7B models — Synology owners don't get that option.

FAQ

Can my ARM-based Synology run Ollama? No. Realtek/Marvell ARM units (DS124, DS223, DS223j) can't run the x86 Ollama image, and lack the memory to be useful even if they could. Only Intel/AMD Plus-series models work.

How much RAM do I need? 6–8 GB free for 3B models, 12–16 GB free for 7B. Most Plus-series units take a cheap SODIMM upgrade to 16–32 GB, which is the best upgrade for this workload.

How fast will it be? Roughly 2–5 tok/s on a Celeron N5095, 5–8 tok/s on Ryzen-based units, for 3B models on CPU. Fine for background tasks; noticeably slower than any cloud chatbot.

Does this work on QNAP too? Yes — same Docker approach via Container Station, and PCIe-slot QNAP models can add an NVIDIA GPU for a huge speedup. The Compose files above work with minor path changes (/share/docker/...).

Can Home Assistant use it? Yes. HA's native Ollama integration points at http://NAS-IP:11434. Make sure OLLAMA_HOST=0.0.0.0 is set, and consider a small tool-capable model tuned for device control.

Is my data private? Completely. Models run on your hardware; no query, document, or chat log leaves your network unless you expose it yourself.

Bottom Line

A Plus-series Synology with 8–16 GB of RAM turns into a genuinely useful private AI endpoint in about 20 minutes: Container Manager, one Compose file, llama3.2:3b, and Open WebUI on port 3000. Keep models at 3B–7B, cap the container's memory, keep port 11434 off the internet, and let the always-on box you already own do quiet AI work in the background — that's the whole trick.