Choose a NAS for Local LLM Repositories

Practical LLM repository tiers

GGUF files, variants and versioning

llama.cpp uses GGUF model files and can fetch compatible quantizations from model repositories. One model family may be retained in multiple quantizations for different machines, plus tokenizer/config assets and previous releases. The repository therefore grows faster than the one file currently loaded.

Fine-tunes and checkpoints can dominate storage during development because intermediate saves accumulate. Set retention rules, preserve immutable release artifacts and deduplicate carefully; deleting the only reproducible checkpoint to save space can cost more than another drive.

HDD versus NVMe and network loading

Large model files are primarily sequential reads, so a healthy HDD array can be an economical source repository. NVMe becomes valuable for concurrent users, many shards, vector indexes and repeated local service restarts. A tiered design keeps the canonical copy and backups on HDD while promoting active models to SSD.

Do not assume a 10GbE label guarantees 1GB/s from every workload. RAID width, drive position, SMB/NFS settings, encryption, small files and client storage all affect transfer speed. Measure end to end and cache locally when startup latency matters.

Model storage decisions