Colibri
Verifiedby JustVugg
Runs large Mixture-of-Experts models such as GLM, DeepSeek V4, and Kimi on consumer hardware by streaming routed experts from SSD, with an OpenAI-compatible local API.
- Access
- CLIAPI
- Auth
- None
- Skills
- 4
Skills
Expert Streaming
Keeps dense weights resident in RAM and streams routed experts from disk on demand, with LRU and pinned hot-store caches.
OpenAI-Compatible API
Serves chat and completion endpoints with SSE streaming and tool calls through coli serve or coli web.
Local Cluster Mode
Spreads routed expert execution across worker machines via a coordinator, keeping routing and KV state local.
Multi-SSD Mirrors
Streams model copies from more than one drive, validating mirrors at startup and falling back to the primary on errors.
Related Agents
ds4
by Salvatore Sanfilippo (antirez)
Runs DeepSeek V4 Flash, GLM 5.x, and Qwen models locally on Metal, CUDA, or ROCm with a native engine, a CLI, a coding…
Magnitude
by Magnitude
Runs open-weight models locally on kernels tuned to your hardware and connects them to coding agents through the…
TensorFold
by ashhart
Serves local language models from a native Zig binary on Apple Silicon and NVIDIA GPUs with OpenAI- and…
vLLM
by vLLM Project
High-throughput LLM inference server with OpenAI-compatible API. Serves Llama, Mistral, Qwen, and 30+ model families…